Browse Source

Python compilation (#19)

* Run compiled analytics script with fallback to regular python

* Update .gitignore

* python compilation docs
pull/1/head
rozetko 6 years ago committed by Alexey Velikiy
parent
commit
38abdcc1ef
  1. 1
      .gitignore
  2. 4
      analytics/.gitignore
  3. 12
      analytics/Compilation.md
  4. 12
      server/src/services/analytics.ts

1
.gitignore vendored

@ -5,7 +5,6 @@ node_modules/
.vscode/
package-lock.json
__pycache__/
*.pyc
*.txt
*.log

4
analytics/.gitignore vendored

@ -0,0 +1,4 @@
build/
dist/
*.spec
__pycache__/

12
analytics/Compilation.md

@ -0,0 +1,12 @@
# Compilation
We use (pyinstaller)[https://www.pyinstaller.org/] to compile analytics unit into binary file supported by *nix systems with all the dependencies.
```bash
pip install pyinstaller
echo "hiddenimports=['pandas._libs.tslibs.timedeltas', 'scipy._lib.messagestream']" | cat > $PYTHON_SITE_PACKAGES/PyInstaller/hooks/hook-pandas.py
cd $HASTIC_SERVER_PATH/analytics
pyinstaller worker.py
```
On Ubuntu 16.04 $PYTHON_SITE_PACKAGES directory located at `~/.local/lib/python3.5/site-packages`

12
server/src/services/analytics.ts

@ -9,9 +9,17 @@ import {
} from './anomalyType'
import { getTarget } from './metrics';
import { getLabeledSegments, insertSegments, removeSegments } from './segments';
import { split, map, mapSync } from 'event-stream';
import { split, mapSync } from 'event-stream';
import * as fs from 'fs';
import * as path from 'path';
const learnWorker = spawn('python3', ['worker.py'], { cwd: ANALYTICS_PATH })
var learnWorker;
if(fs.existsSync(path.join(ANALYTICS_PATH, 'dist/worker/worker'))) {
learnWorker = spawn('dist/worker/worker', [], { cwd: ANALYTICS_PATH })
} else {
// If compiled analytics script doesn't exist - fallback to regular python
learnWorker = spawn('python3', ['worker.py'], { cwd: ANALYTICS_PATH })
}
learnWorker.stdout.pipe(split()).pipe(mapSync(onMessage));
learnWorker.stderr.on('data', data => console.error(`worker stderr: ${data}`));

Loading…
Cancel
Save