NDJSON

NDJSON

The namespace for NDJSON utility functions.

Source:

Methods

(static) diff(arg0_id, arg1_timestamp) → {Promise.<(Object|null)>}

Returns a diff over .history.keyframes for the ID in question.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_timestamp Object
Returns:
Type:
Promise.<(Object|null)>

(static) diffAll(arg0_timestamp) → {Promise.<Array.<Object>>}

Diffs all .history.keyframes for all Objects for a given ID, so long as they have that field.

Source:
Parameters:
Name Type Description
arg0_timestamp number | string
Returns:
Type:
Promise.<Array.<Object>>

(static) getDiagnostics() → {Promise.<Array.<{worker_id: number, rss: number, heapUsed: number, heapTotal: number, heapLimit: number, percentage: number}>>}

Resolves active RAM diagnostic percentage statistics from every worker in the pool.

Source:
Returns:
Type:
Promise.<Array.<{worker_id: number, rss: number, heapUsed: number, heapTotal: number, heapLimit: number, percentage: number}>>

(static) getDiffs(arg0_ids, arg1_timestamp) → {Promise.<Array.<(Object|null)>>}

Returns diffs over .history.keyframes for multiple IDs in 1-pass.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
arg1_timestamp number
Returns:
Type:
Promise.<Array.<(Object|null)>>

(static) getKeyframes(arg0_id) → {Promise.<Object>}

Returns processed .history.keyframes for a given key.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<Object>

(static) getValue(arg0_id) → {Promise.<Object>}

Returns the Object value of a single ID.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<Object>

(static) getValues(arg0_ids) → {Promise.<Array.<(Object|null)>>}

Returns the Object values of multiple IDs in 1-pass.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
Returns:
Type:
Promise.<Array.<(Object|null)>>

(static) getWorkerID(arg0_id, arg1_pool_length) → {number}

Returns the Worker ID that holds a particular ID's partition.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_pool_length number
Returns:
Type:
number

(static) getWorkerPool(arg0_max_workersopt) → {Array.<NodeWorker>}

Returns the current NDJSON worker pool managing DBs.

Source:
Parameters:
Name Type Attributes Default Description
arg0_max_workers number <optional>
os.cpus().length - 1
Returns:
Type:
Array.<NodeWorker>

(static) load(arg0_file_path, arg1_optionsopt) → {Promise.<unknown>}

Loads a regular JSON file and partitions it into NDJSON files.

Source:
Parameters:
Name Type Attributes Description
arg0_file_path string
arg1_options Object <optional>
Name Type Attributes Default Description
dynamic_chunk_size number <optional>
67108864

64MB

dynamic_max_workers number <optional>
os.cpus().length - 1
ram_threshold number <optional>

% Threshold of RAM dedicated to RAM queries.

Returns:
Type:
Promise.<unknown>

(static) partitionFile(arg0_file_path) → {Promise.<void>}

Partitions a given file into multiple NDJSON files for use. Internal helper function.

Source:
Parameters:
Name Type Description
arg0_file_path string
Returns:
Type:
Promise.<void>

(static) processSegment()

Processes a segment. Internal helper function for parsing NDJSON.

Source:

(static) query(arg0_optionsopt) → {Promise.<Array.<Object>>}

Queries an NDJSON file. [WIP] - Should be refactored so that only arg1_options is present.

Source:
Parameters:
Name Type Attributes Description
arg0_options Object <optional>
Name Type Attributes Default Description
limit_end number <optional>
limit_start number <optional>
0
query_obj Object <optional>

Key/value pairs to match for.

Returns:
Type:
Promise.<Array.<Object>>

(static) removeValue(arg0_id) → {Promise.<boolean>}

Removes a value from the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<boolean>

(static) removeValues(arg0_ids) → {Promise.<boolean>}

Removes multiple values from the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
Returns:
Type:
Promise.<boolean>

(static) save() → {Promise.<void>}

Saves the NDJSON file back into the main directory.

Source:
Returns:
Type:
Promise.<void>

(static) setKeyframes(arg0_id, arg1_keyframes) → {Promise.<boolean>}

Sets the .history.keyframes object for a given key.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_keyframes Object
Returns:
Type:
Promise.<boolean>

(static) setValue(arg0_id, arg1_value) → {Promise.<boolean>}

Sets a key-value pair in the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_value Object
Returns:
Type:
Promise.<boolean>

(static) setValues(arg0_update_map) → {Promise.<boolean>}

Sets multiple key-value pairs for the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_update_map Object
Returns:
Type:
Promise.<boolean>

(static) task(arg0_target, arg1_message) → {Promise.<(any|Array.<any>)>}

Internal dispatcher to handle messaging and promise queues across worker pools.

Source:
Parameters:
Name Type Description
arg0_target number | Array.<number> | string

Target worker ID, array of IDs, partition ID string, or "all".

arg1_message Object | function

Message payload, or a factory function returning payload.

Returns:
Type:
Promise.<(any|Array.<any>)>

NDJSON

The namespace for NDJSON utility functions.

Source:

Methods

(static) diff(arg0_id, arg1_timestamp) → {Promise.<(Object|null)>}

Returns a diff over .history.keyframes for the ID in question.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_timestamp Object
Returns:
Type:
Promise.<(Object|null)>

(static) diffAll(arg0_timestamp) → {Promise.<Array.<Object>>}

Diffs all .history.keyframes for all Objects for a given ID, so long as they have that field.

Source:
Parameters:
Name Type Description
arg0_timestamp number | string
Returns:
Type:
Promise.<Array.<Object>>

(static) getDiagnostics() → {Promise.<Array.<{worker_id: number, rss: number, heapUsed: number, heapTotal: number, heapLimit: number, percentage: number}>>}

Resolves active RAM diagnostic percentage statistics from every worker in the pool.

Source:
Returns:
Type:
Promise.<Array.<{worker_id: number, rss: number, heapUsed: number, heapTotal: number, heapLimit: number, percentage: number}>>

(static) getDiffs(arg0_ids, arg1_timestamp) → {Promise.<Array.<(Object|null)>>}

Returns diffs over .history.keyframes for multiple IDs in 1-pass.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
arg1_timestamp number
Returns:
Type:
Promise.<Array.<(Object|null)>>

(static) getKeyframes(arg0_id) → {Promise.<Object>}

Returns processed .history.keyframes for a given key.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<Object>

(static) getValue(arg0_id) → {Promise.<Object>}

Returns the Object value of a single ID.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<Object>

(static) getValues(arg0_ids) → {Promise.<Array.<(Object|null)>>}

Returns the Object values of multiple IDs in 1-pass.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
Returns:
Type:
Promise.<Array.<(Object|null)>>

(static) getWorkerID(arg0_id, arg1_pool_length) → {number}

Returns the Worker ID that holds a particular ID's partition.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_pool_length number
Returns:
Type:
number

(static) getWorkerPool(arg0_max_workersopt) → {Array.<NodeWorker>}

Returns the current NDJSON worker pool managing DBs.

Source:
Parameters:
Name Type Attributes Default Description
arg0_max_workers number <optional>
os.cpus().length - 1
Returns:
Type:
Array.<NodeWorker>

(static) load(arg0_file_path, arg1_optionsopt) → {Promise.<unknown>}

Loads a regular JSON file and partitions it into NDJSON files.

Source:
Parameters:
Name Type Attributes Description
arg0_file_path string
arg1_options Object <optional>
Name Type Attributes Default Description
dynamic_chunk_size number <optional>
67108864

64MB

dynamic_max_workers number <optional>
os.cpus().length - 1
ram_threshold number <optional>

% Threshold of RAM dedicated to RAM queries.

Returns:
Type:
Promise.<unknown>

(static) partitionFile(arg0_file_path) → {Promise.<void>}

Partitions a given file into multiple NDJSON files for use. Internal helper function.

Source:
Parameters:
Name Type Description
arg0_file_path string
Returns:
Type:
Promise.<void>

(static) processSegment()

Processes a segment. Internal helper function for parsing NDJSON.

Source:

(static) query(arg0_optionsopt) → {Promise.<Array.<Object>>}

Queries an NDJSON file. [WIP] - Should be refactored so that only arg1_options is present.

Source:
Parameters:
Name Type Attributes Description
arg0_options Object <optional>
Name Type Attributes Default Description
limit_end number <optional>
limit_start number <optional>
0
query_obj Object <optional>

Key/value pairs to match for.

Returns:
Type:
Promise.<Array.<Object>>

(static) removeValue(arg0_id) → {Promise.<boolean>}

Removes a value from the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_id string
Returns:
Type:
Promise.<boolean>

(static) removeValues(arg0_ids) → {Promise.<boolean>}

Removes multiple values from the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_ids Array.<string>
Returns:
Type:
Promise.<boolean>

(static) save() → {Promise.<void>}

Saves the NDJSON file back into the main directory.

Source:
Returns:
Type:
Promise.<void>

(static) setKeyframes(arg0_id, arg1_keyframes) → {Promise.<boolean>}

Sets the .history.keyframes object for a given key.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_keyframes Object
Returns:
Type:
Promise.<boolean>

(static) setValue(arg0_id, arg1_value) → {Promise.<boolean>}

Sets a key-value pair in the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_id string
arg1_value Object
Returns:
Type:
Promise.<boolean>

(static) setValues(arg0_update_map) → {Promise.<boolean>}

Sets multiple key-value pairs for the NDJSON file.

Source:
Parameters:
Name Type Description
arg0_update_map Object
Returns:
Type:
Promise.<boolean>

(static) task(arg0_target, arg1_message) → {Promise.<(any|Array.<any>)>}

Internal dispatcher to handle messaging and promise queues across worker pools.

Source:
Parameters:
Name Type Description
arg0_target number | Array.<number> | string

Target worker ID, array of IDs, partition ID string, or "all".

arg1_message Object | function

Message payload, or a factory function returning payload.

Returns:
Type:
Promise.<(any|Array.<any>)>