Hadoop 2.5.2 mvn构建失败Windows 7 x64 - hadoop-common上的编译ms-native-dll？

Question

我遵循Hadoop2OnWindows Wiki here，并且我多次阅读源树中的BUILDING.txt。我想我已经找到了所有的依赖关系，但是我得到了一个奇怪的错误（下面的完整日志）Failed to execute goal (compile-ms-native-dll)。

我已经安装了Windows SDK 7.1

下面的事情是在我的PATH：

• Cygwin/bin
• 的MSBuild（C:\Windows\Microsoft.NET\Framework64\v4.0.30319）
• 协议缓冲区2.5.0
• 的Zlib和zlib/include（以便它可以访问zlib.h）
• Findbugs
• Maven的

我还设置Platform=x64，不用说，我JAVA_HOME和M2_HOME也设置。

我在Windows SDK 7.1命令提示符下运行mvn package -Pdist,native-win,docs -DskipTests -Dtar。

我早些时候有Gnuwin32这给了我下面的错误，但移动到Cygwin/bin怀疑可能是Gnuwin32的不妥之处，但我仍然得到以下错误。我该怎么办？

这里的错误日志，

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (com 
pile-ms-native-dll) on project hadoop-common: Command execution failed. Process 
exited with an error: 1(Exit value: 1) -> [Help 1] 
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal o 
rg.codehaus.mojo:exec-maven-plugin:1.2:exec (compile-ms-native-dll) on project h 
adoop-common: Command execution failed. 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:216) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:153) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:145) 
     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje 
ct(LifecycleModuleBuilder.java:116) 
     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje 
ct(LifecycleModuleBuilder.java:80) 
     at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThre 
adedBuilder.build(SingleThreadedBuilder.java:51) 
     at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lifecycl 
eStarter.java:120) 
     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347) 
     at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154) 
     at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582) 
     at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214) 
     at org.apache.maven.cli.MavenCli.main(MavenCli.java:158) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. 
java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces 
sorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Laun 
cher.java:289) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav 
a:229) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La 
uncher.java:415) 
     at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java: 
356) 
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution fai 
led. 
     at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:345) 
     at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(Default 
BuildPluginManager.java:132) 
     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor 
.java:208) 
     ... 19 more 

这只是在hadoop-common，它的失败。这里有一些更多的日志可能是有用的，我只是无法弄清楚它是什么，虽然，

[INFO] 
[INFO] --- exec-maven-plugin:1.2:exec (compile-ms-native-dll) @ hadoop-common -- 
- 
[DEBUG] Configuring mojo org.codehaus.mojo:exec-maven-plugin:1.2:exec from plugi 
n realm ClassRealm[plugin>org.codehaus.mojo:exec-maven-plugin:1.2, parent: sun.m 
[email protected]] 
[DEBUG] Configuring mojo 'org.codehaus.mojo:exec-maven-plugin:1.2:exec' with bas 
ic configurator --> 
[DEBUG] (f) arguments = [C:\hdc\hadoop-common-project\hadoop-common/src/main/n 
ative/native.sln, /nologo, /p:Configuration=Release, /p:OutDir=C:\hdc\hadoop-com 
mon-project\hadoop-common\target/bin/, /p:CustomSnappyPrefix=, /p:CustomSnappyLi 
b=, /p:CustomSnappyInclude=, /p:RequireSnappy=false] 
[DEBUG] (f) basedir = C:\hdc\hadoop-common-project\hadoop-common 
[DEBUG] (f) classpathScope = runtime 
[DEBUG] (f) executable = msbuild 
[DEBUG] (f) longClasspath = false 
[DEBUG] (f) project = MavenProject: org.apache.hadoop:hadoop-common:2.5.2 @ C: 
\hdc\hadoop-common-project\hadoop-common\pom.xml 
[DEBUG] (f) session = [email protected] 
[DEBUG] (f) skip = false 
[DEBUG] -- end configuration -- 
[DEBUG] Executing command line: msbuild C:\hdc\hadoop-common-project\hadoop-comm 
on/src/main/native/native.sln /nologo /p:Configuration=Release /p:OutDir=C:\hdc\ 
hadoop-common-project\hadoop-common\target/bin/ /p:CustomSnappyPrefix= /p:Custom 
SnappyLib= /p:CustomSnappyInclude= /p:RequireSnappy=false 
Building the projects in this solution one at a time. To enable parallel build, 
please add the "/m" switch. 
Build started 23-11-2014 00:07:43. 
Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" 
on node 1 (default targets). 
ValidateSolutionConfiguration: 
    Building solution configuration "Release|x64". 
Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" 
(1) is building "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\nati 
ve.vcxproj" (2) on node 1 (default targets). 
InitializeBuildStatus: 
    Touching "..\..\..\target\native\Release\native.unsuccessfulbuild". 
ClCompile: 
    C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\CL.exe /c /I. 
.\winutils\include /I..\..\..\target\native\javah /IC:\Java\jdk1.7.0_71\include 
/IC:\Java\jdk1.7.0_71\include\win32 /I.\src /Zi /nologo /W3 /WX- /O2 /Oi /GL /D 
WIN32 /D NDEBUG /D _WINDOWS /D _USRDLL /D NATIVE_EXPORTS /D _WINDLL /D _UNICODE 
/D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fo"..\.. 
\..\target\native\Release\\" /Fd"..\..\..\target\native\Release\vcWindows7.1SDK. 
pdb" /Gd /TC /wd4244 /errorReport:queue src\org\apache\hadoop\io\compress\zlib\Z 
libCompressor.c src\org\apache\hadoop\io\compress\zlib\ZlibDecompressor.c 
    ZlibCompressor.c 
c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop 
\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C 
annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm 
on-project\hadoop-common\src\main\native\native.vcxproj] 
    ZlibDecompressor.c 
c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop 
\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C 
annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm 
on-project\hadoop-common\src\main\native\native.vcxproj] 
Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ 
e\native.vcxproj" (default targets) -- FAILED. 
Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ 
e\native.sln" (default targets) -- FAILED. 

Build FAILED. 

"C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default 
target) (1) -> 
"C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (def 
ault target) (2) -> 
(ClCompile target) -> 
    c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado 
op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: 
Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co 
mmon-project\hadoop-common\src\main\native\native.vcxproj] 
    c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado 
op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: 
Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co 
mmon-project\hadoop-common\src\main\native\native.vcxproj] 

    0 Warning(s) 
    2 Error(s) 

Answer 1

从日志记录，显然，你错过zlib.h 你设置的环境变量ZLIB_HOME包含zlib.h目录？ 注意把zlib.h目录路径是错误的，没有必要的，不仅需要zlib的bin目录，在大厦的导游说https://svn.apache.org/viewvc/hadoop/common/branches/branch-2/BUILDING.txt?view=markup

此外，zlib.h需要一些头不存在在Windows中，所以你需要下载这些头文件，并将其放入与zlib.h相同的文件夹中。 有关详细信息，请参阅Is there a replacement for unistd.h for Windows (Visual C)?得到unistd.h中， 和https://gist.github.com/ashelly/7776712得到getopt.h

对于那些乡亲在Win32（像我一样），你可能会面临更多的麻烦。关键的解决方案应该是编辑winutils和native package的.sln和.vcxprj文件，以便它们与Win32平台兼容。

Answer 2

我个人使用的命令mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true成功地在Windows 8版本构建2.4.1我有许多不同的问题，但不记得这是其中之一。但它似乎抱怨zlib。

Answer 3

下载的protobuf并将其添加到您的PATH

https://code.google.com/p/protobuf/downloads/detail?name=protoc-2.5.0-win32.zip&can=2&q=