DEX8-SDK CLI

Simple DEX8-SDK Command Line Interface to
initiate, delete, start, upload and download tasks
DEX8-SDK CLI is a command line interface which developers can use in terminal. It is a very handy tool to start a new crawler, spider, scraper or any new automation project and to integrate it with the DEX8 platform.

Installation

NPM Package: https://www.npmjs.com/package/dex8-sdk
It is recommended to install it globally:
npm install -g dex8-sdk

In Linux OS do not forget to use sudo command.
When installed properly, DEX8 CLI commands are ready.

CLI Commands

CLI Commands are used in the OS terminal like Linux GNOME Terminal, Windows Shell or any other.

$ dex8 init <taskName>

Use this command to start a new project (DEX8 Task). It will create a taskName folder with initial files. DEX8 Task File Structure
a) Move to that directory to continue using dex8 commands: $ cd taskName
b) Open "manifest.json" file and modify title to taskName.

$ dex8 login

Login to DEX8 Web Panel with your username and password. After successfully logging in, a "conf.js" file is created. It contains a JWT Authentication Token, database credentials and other sensitive data so please include it in .gitignore and NEVER push this file into the git repository.

Specific Task Login

In most cases for logging in you need to position in the task folder. For example: $ cd task1 and then $ dex8 login. After successfully logging in the file "conf.js" will be created under task folder (task1) and that conf will be used for that specific task.
DEX8 Specific Task Login

Mutual Login

It's also possible to mutually login if you are using the same login for many tasks under the projects folder. In that case create a position in the projects folder and login with $ dex8 login command. A "conf.js" file will be created in the projects folder and it will be mutual for all tasks under the projects folder.
DEX8 Mutual Login

Notice: A "conf.js" file in task folder has higher priority and will be used instead of a "conf.js" file in the projects folder.

$ dex8 logout

Logout from DEX8 Web Panel.
It is highly recommended to logout every time development work is finished because it will delete the "conf.js" file.

$ dex8 delete <taskName>

Delete a DEX8 task. This command will delete the whole taskName folder. Curent working directory can be either in taskName or in its upper, parent directory.

Aliases:

$ dex8 rm <taskName>

$ dex8 start

When a task is created use this command to run a DEX8 task locally. Echo messages will be printed in terminal. When the task is started in Web Panel the same messages will appear in Echoes » Real-time section. DEX8 Echoes

Options:

-i --input  <inputFile.js>      select input file (initial data for DEX8 task)
-s --short                      prints short messages

$ dex8 upload

Upload task to the web panel.
DEX8 Task Upload

After the successful upload the task should appear in Web Panel under Tasks section. DEX8 Task List

Aliases:

$ dex8 u <taskName>
Options:
To use options position current working directory to folder above taskName.
-t --task  <taskName>      upload task by task name
-a --all                       upload all tasks

$ dex8 update

Update task details without uploading the tasks.
This command will read what is written in "manifest.json" and "howto.html" and update the task.
Althgough same can be done with "$dex8 upload" this is much faster because it will not change files.
The position have to be in the task's folder.

$ dex8 download <task_id>

Download task files by task_id. Parameter task_id can be found in Web Panel - Tasks table - TID column.
This command will first delete all files in the folder and then create new, downloaded files.
Login is required before using this command e.g. "conf.js" file must be created. DEX8 Taask Id TID

Aliases:

$ dex8 d





Runtime Commands new

DEX8-SDK Runtime Commands are used to control DEX8 task during runtime i.e. while NodeJS process is running. Once the task is started with $ dex8 start -i input.json it can be paused, resumed or stoped.
Also it's very useful to test particular function without the need of starting the task from the beggining every time we want to test that function. This can save a lot of developer hours.

Implemented after dex8-sdk version 2.9.0 !

p

Use this command to pause already started task.
Notice: This command will not pause currently running function. That function will be executed till the end and the next function will be paused. For example if we have ff.serial([f1, f2, f3]) and p command is executed during f2 runtime, then f2 will finish but f3 will not start.

r

Resume the paused task. Usually it's used after command p when the paused task should be started again.

s

Stop the running task. When this command is used the current function will be executed till the end and all next functions will not be executed.

k

Kill the running NodeJS process and exit to the terminal command prompt. It's simmilar to CTRL+C.

input.json

When the command contains "input" and ".json" at the end it will reload the input file. For example myInput5.json will load new input data and the task will continue to work with that data.
It's useful when we want to see how different inputs will reflect to the function execution.

Important: Add input into the lib parameter with ff.libAdd({input}); in your main.js file.

i

Show the input which is currently in use.
Usually shows the input used by CLI command $dex8 start -i input.json or reloaded by the runtime command input2.json

x

Show the "x" transitional variable in the FunctionFlow.

x.field

Add new or modify existing x field value.

e

Evaluate some Javascript code. It's simmilar to command $ node and it's useful when developer needs to test some JS code snippet quickly.

f

Test the FunctionFlow function code "ff_code" which has two parameters "x" and "lib" (x, lib) => { ...ff_code... }. Usually before using this command it's needed to pause the task with p.
Developers can use this command to make quick tests on the code inside FF function with the ability to use parameters "x" and "lib" which are usually inserted in the main.js file with:

main.js
----------------
ff.setOpts({debug: false, msDelay: 5500});
ff.xInject(x);
ff.libInject(lib);
ff.libAdd({proxy, pupconfig, input});

f1 [,f2, ...]

If the command doesn't correspond to any of the previous commands then it will search for the function file to execute it. For example if the command is login it will search for "login.js" and if exists it will execute it. Also it's possible to define multiple functions to be executed serially ff.serial[f1, f2, f3].
Usually before using this command it's needed to pause the task with p.



Video example on DEX8-SDK Runtime Commands