thingsboard develop environment set up

I. dependency

• git

  版本管理工具,注意配置环境变量，保证命令窗口能够直接使用。

• oracle jdk

  Which Java package do I need?
  
  Software Developers: JDK (Java SE Development Kit). For Java Developers. Includes a complete JRE plus tools for developing, debugging, and monitoring Java applications.
  Administrators running applications on a server:  Server JRE (Server Java Runtime Environment) For deploying Java applications on servers. Includes tools for JVM monitoring and tools commonly required for server applications, but does not include browser integration (the Java plug-in), auto-update, nor an installer. Learn more arrow
  End user running Java on a desktop: JRE: (Java Runtime Environment). Covers most end-users needs. Contains everything required to run Java applications on your system.

  警告：需要的jdk，非jre。选择下载jdk1.8，如果是jdk13编译会报错。

• node.js

  现在安装成功后，需要配置环境变量。保证在cmd窗口node能够直接运行，并且通过node -v 查询版本后修改thingsboa()rd 工程中的pom.xml。

• maven

  工程是基于maven管理，直接通过idea open，之后会自动下载各种依赖包。

  C:\Users\yuren\.m2\repository

  $tree ~/.m2 -L 2
  /home/jay/.m2
  └── repository
      ├── antlr
      ├── aopalliance
      ├── asm
      ├── backport-util-concurrent
      ├── ch
      ...

  按需设置maven镜像源，否则下载速度可能不稳定。在maven安装包目录下找到settings.xml更改，详细参考，https://developer.aliyun.com/mvn/guide。

  <mirrors>
      <mirror>
          <id>aliyunmaven</id>
          <mirrorOf>*</mirrorOf>
          <name>阿里云公共仓库</name>
          <url>https://maven.aliyun.com/repository/public</url>
      </mirror>
  </mirrors>

  提示：linux在/etc/maven/settings.xml， win在zip解压安装包内查找。

• npm

  npm --registry https://registry.npm.taobao.org install express #临时使用
  npm config set registry https://registry.npm.taobao.org        #持久使用
  npm config get registry                                        #查询
  npm config rm registry                                         #删除
  npm info express

• idea intellj

  集成开发环境，内部集成maven。

• docker
• postgresql

  D:\my_project\thingsboard\dao\src\main\resources\sql

linux

sudo apt-get install openjdk-8-jdk maven

II. install

• Building from sources
• Contribution Guide

git clone 整个thingsboard 工程。

git clone https://github.com/thingsboard/thingsboard.git

成功clone 工程后目录结构如下。

$tree thingsboard/ -L 1
thingsboard/
├── application
├── common
├── dao
├── docker
├── img
├── k8s
├── LICENSE
├── license-header-template.txt
├── msa
├── netty-mqtt
├── pom.xml                         #maven 工程配置文件
├── README.md
├── rule-engine
├── tools
├── transport 
└── ui                               #web ui项目

这里阐述thingboard在win下的开发环境快速建立，包含工程建立、编译、调试环境。

• 装jdk、maven、node、git并且注意配置环境变量，确定命令能够直接运行，同时设置maven、node镜像源；

• 通过maven编译整个工程，下载依赖，编译protobuf 文件，java文件；

  mvn clean install
  #mvn clean install -Ddockerfile.skip=false
  mvn clean install -DskipTests           #如果不跳过测试，编译时间可能会超过1h

• 通过idea itellij import maven工程，编辑，调试；

  

III. troubleshooting

maven:Some files do not have the expected license header

[ERROR] Failed to execute goal com.mycila:license-maven-plugin:3.0:check (default) on project thingsboard: Some files do not have the expected license header -> [Help 1]

license 检查不通过。

  <plugins>
      <plugin>
      <groupId>com.mycila</groupId>
      <artifactId>license-maven-plugin</artifactId>
      </plugin>
 </plugins>

maven:Could not resolve dependencies for project org.thingsboard:application:jar:2.4.1

[ERROR] Failed to execute goal on project application: Could not resolve dependencies for project org.thingsboard:application:jar:2.4.1: The following artifacts could not be resolved: org.thingsboard.rule-engine:rule-engine-components:jar:2.4.1, org.thingsboard:dao:jar:2.4.1: Could not find artifact org.thingsboard.rule-engine:rule-engine-components:jar:2.4.1 in jenkins (http://repo.jenkins-ci.org/releases) -> [Help 1]

根目录下去maven编译，不要单独每个模块，不能自动解决依赖。

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project dao: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /home/jay/Desktop/project/thingsboard/dao && /usr/lib/jvm/zulu-8-amd64/jre/bin/java -jar /home/jay/Desktop/project/thingsboard/dao/target/surefire/surefirebooter2640713033967879111.jar /home/jay/Desktop/project/thingsboard/dao/target/surefire/surefire360942185575801173tmp /home/jay/Desktop/project/thingsboard/dao/target/surefire/surefire_07635717346452894211tm

Failed to run task: 'npm install' failed

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.0:npm (npm install) on project ui: Failed to run task: 'npm install' failed. (error code 1) -> [Help 1]

修改版本号，保持和当前node、npm一致。

通过npm --verbose install

git Refusing to remove it.

npm ERR! path /home/jay/Desktop/project/thingsboard/ui/node_modules/mdPickers
npm ERR! code EISGIT
npm ERR! git /home/jay/Desktop/project/thingsboard/ui/node_modules/mdPickers: Appears to be a git repo or submodule.
npm ERR! git     

npm ERR! git Refusing to remove it. Update manually,
npm ERR! git or move it out of the way first.

ui/node_modules/ 删除后再执行install。

手动删除

rm -r /home/jay/Desktop/project/thingsboard/ui/node_modules/mdPickers/

java:变量 log找不到符号

Error:(88, 21) java: 找不到符号
  符号:   变量 log
  位置: 类 org.thingsboard.server.common.data.SearchTextBasedWithAdditionalInfo<I>

使用jdk1.8 编译，先前使用了13

java:类org.thingsboard.server.common.msg.gen 和org.thingsboard.server.gen.cluster 不存在。

Error:(25, 45) java: 程序包org.thingsboard.server.common.msg.gen不存在
Error:(22, 42) java: 程序包org.thingsboard.server.gen.cluster不存在

这些类是通过protobuf 自动生成的，需要在执行java编译前进行maven clean install

maven:Failed to delete tb-http-transport.rpm

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on project http: Failed to clean project: Failed to delete D:\my_project\thingsboard\transport\http\target\tb-http-transport.rpm -> [Help 1]

第一次编译失败，再次编译可能会提示该错误，可以手动到报错路径删除，如果提示文件正在使用，需要在任务管理器杀死java进程后再手动删除。

maven:Failed to delete node.exe

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-clean-plugin:2.5:clean (default-clean) on project ui: Failed to clean project: Failed to delete D:\my_project\thingsboard\ui\target\node\node.exe -> [Help 1]

任务管理器手动杀死node.exe。

npm:npm:cb() never called!

npm install 错误

npm ERR! cb() never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR!     <https://npm.community>

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\yuren\AppData\Roaming\npm-cache\_logs\2019-11-06T10_55_28_258Z-debug.log

• 尝试npm cache clean --force 后再次npm install 无果；
• 尝试更换淘宝镜像源后再次npm install 无果；
• 怀疑有些包下载需要翻墙，全局代理翻墙后问题依然；
• 参考网上关闭所有代理后问题依然；

• 通过log日志分析最后一个解包报错的地方，屏蔽需要的material-design-icons，新modules rxjs仍然报错；

  extract material-design-icons@3.0.1 extracted to node_modules\.staging\material-design-icons-61b4d55e (72881ms)

  extract rxjs@6.5.2 extracted to node_modules\.staging\rxjs-e901ba4c (24280ms)

• 参考npm ERR cb() never called 执行

  npm install --no-package-lock

  之后提示npm ERR! path git，添加git 到环境变量后正常。

npm:npm ERR! path git

npm ERR! path git
npm ERR! code ENOENT
npm ERR! errno ENOENT
npm ERR! syscall spawn git
npm ERR! enoent Error while executing:
npm ERR! enoent undefined ls-remote -h -t git://github.com/fabiobiondi/angular-

添加git到环境变量。

maven:Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project dao

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project dao: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/jay/Desktop/project/thingsboard/dao/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 

ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.0:npm (npm install) on project ui: Failed to run task: 'npm install' failed. (error code 1) -> [Help 1]

3这里在linux上面编译报错，先前通过docker启动了thingsboard服务，停掉后再编译正常。

ThingsBoard Web UI Microservice

[ERROR] Unhandled rejection Error: EACCES: permission denied, mkdir '/home/jay/.npm/_cacache/index-v5/34/c5'
[ERROR] npm ERR! cb() never called!
[ERROR] 
[ERROR] npm ERR! This is an error with npm itself. Please report this error at:
[ERROR] npm ERR!     <https://npm.community>

[ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:1.0:npm (npm install) on project web-ui: Failed to run task: 'npm install' failed. (error code 1) -> [Help 1]

该目录变成变了root用户，估计是先前通过sudo npm install 新建的，改进改了回来，看来sudo还真不能随便用。

ls -l /home/jay/.npm/_cacache/index-v5|grep 34
drwxr-xr-x 17 root root 4096 11月  6 14:41 34

sudo chown -R jay:jay /home/jay/.npm/_cacache/index-v5/34

ThingsBoard Server Application ..................... FAILURE [13:45 min]

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project application: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_231\jre\bin\java" -jar D:\my_project\thingsboard\application\target\surefire\surefirebooter1013763599726310635.jar D:\my_project\thingsboard\application\target\surefire\surefire5278641689999662714tmp D:\my_project\thingsboard\application\target\surefire\surefire_12648731641714611711tmp"
[ERROR] -> [Help 1]

看起来像是运行时错误，这里尝试重启后再次编译。

No compiler is provided in this environment

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.
1:compile (default-compile) on project netty-mqtt: Compilation failure
[ERROR] No compiler is provided in this environment. Perhaps you are running on
a JRE rather than a JDK?

需要在环境变量中设置java，包含%JAVA_HOME%bin;%JAVA_HOME%lib;