High-level FLM content processing

import { ZooFLMProcessor } from '@phfaist/zoodb/dbprocessor/flmprocessor';

The high-level FLM processor will automatically scan FLM content for labels, citations, external resources, etc. It will also create a citation manager and a citation compiler/formatter. It uses the FLMSimpleContentCompiler() DB processor under the hood.

class ZooFLMProcessor()

Database processor that compiles FLM content in object fields while providing the full infrastructure to handle cross-references, external resources, and citations (fetching, compiling, formatting).

The constructor options object recognises the following properties:

  • zoo_flm_environment (required) — the ZooFLMEnvironment() instance to use.

  • refs — per-object-type override options for reference registration, keyed by object type. Each value is an object that may contain formatted_ref_flm_text_fn(objid, obj) => string, which controls the display text for cross-references to objects of that type.

  • refs_defaults — default override options applied to all object types before any per-type override in refs.

  • citations.sources — an object mapping citation prefixes to CitationSourceBase() instances (e.g. { arxiv: new CitationSourceArxiv(...) }).

  • citations.csl_style — the CSL style XML string used by CitationCompiler() to format citations.

  • citations.default_user_agent — HTTP User-Agent header for citation source requests.

  • citations.cache_fs — a filesystem object for cache file access.

  • citations.cache_dir — directory path for the two citation cache files (cache_downloaded_info.json and cache_compiled_citations.json). Defaults to '.'.

  • citations.cache_entry_default_duration_ms — how long fetched citation information remains valid in the cache (milliseconds).

  • citations.skip_save_cache — when true, the citation caches are not written to disk. Useful in test environments.

  • citations.citation_manager_options — additional options forwarded directly to CitationDatabaseManager().

  • flm_error_policy — 'abort' (default) or 'continue'. Controls behaviour when an FLM compilation error occurs.

  • skip_check_update_existing_citations — when true, citations already present in the environment’s citations provider are not re-fetched or re-compiled. Useful for incremental updates.

  • skip_check_update_existing_resources — when true, graphics resources already registered in the environment’s graphics collection are not re-collected.

  • resource_collector — provide a ResourceCollector() instance directly; when set, resource_collector_options is ignored.

  • resource_collector_options — options forwarded to the ResourceCollector() constructor if resource_collector is not provided.

In normal usage this class is instantiated by use_flm_processor() which constructs all the citation sources and resource collector for you.

ZooFLMProcessor.ZooFLMProcessor
ZooFLMProcessor.compile_flm_fragment(flm_content)

You can call this method to compile additional, ad hoc fragments that do not appear as object field values.

If you compile additional fragments, don’t forget to call process_auxiliary_resources() afterwards to register any new citations, cross-reference targets, and external resources found in the new fragments.

ZooFLMProcessor.process_auxiliary_resources()

You should call this method if you compile more fragments using compile_flm_fragment(). This method will ensure that any additional citations are collected, any referenceable targets are registered, and any additional resources are collected.

Low-level FLM content object field compilation

import {
    FLMSimpleContentCompiler
} from '@phfaist/zoodb/dbprocessor/flmsimplecontent';

This database processor will only take care of converting FLM content fields of objects into compiled FLM fragment objects. Optionally, if a ZooFLMScanner() is provided, the field values are scanned as they are being compiled to collect references, citations, and resources.

class FLMSimpleContentCompiler()

A class that can compile FLM content fields into FLM fragments.

Configuration options (config argument):

  • flm_environment - the environment object to use to create fragments

  • content_scanner - if set, it is assumed to be a ZooFLMScanner() instance, and all compiled FLM fragments are scanned.

  • flm_error_policy - one of ‘abort’ (the default) or ‘continue’. If ‘abort’, then any errors that arise when compile FLM content cause a JS exception/error to be thrown. If ‘continue’, then a warning is triggered, and the fragment contents is set to a text that contains a human-readable description of the error. (The ‘continue’ is meant for use, for instance, in a visual editor application that provides an instant preview of the typed FLM code.)

Parsing of DB fields follows the schema’s _flm field. The value of that field can either be a dictionary, a string, true, false, or null. If it’s a dictionary, then it should be of the form { enabled: true|false, standalone: true|false, is_block_level: true|false|null } (all fields default to false if omitted, except is_block_level which defaults to null). The schema’s _flm field can also be true or 'full' (shorthands for { enabled: true, standalone: false }), 'standalone' (a shorthand for { enabled: true, standalone: true }), 'block_level' (a shorthand for { enabled: true, standalone: false, is_block_level: true }), or false or null (interpreted as { enabled: false }).

If enabled is true, then FLM processing is enabled for that field, otherwise it is disabled. If standalone is true, then any FLM processing of that field happens in standalone mode. Standalone mode = content that can be rendered in any context = content that doesn’t make use of citations, cross-references or similar context-dependent content. Standalone mode is useful for pieces of text that you’d want to be able to render in multiple different contexts (e.g. an object’s name or title, which you might want to reproduce in links to that object or in a list of objects).

If is_block_level is true, then the FLM content is compiled in block-level mode: the content can contain paragraphs, bullet lists, figures, and other block-level constructs. When is_block_level is null (the default), the FLM parser decides based on the content itself.

FLMSimpleContentCompiler.FLMSimpleContentCompiler
FLMSimpleContentCompiler.compile_flm(flm_content)

Compile the given FLM content flm_content according to the options flm_options.

The fragment’s resource_info and what properties are set according to object_type, object, and fieldname. We’ll inspect object._zoodb to get information like object ID, source file name, etc.

The newly created FLM fragment is returned. The object itself and its properties are NOT modified.

The FLM options should be an object as is returned by parse_schema_flm_options(). It specifies whether or not the fragment is parsed in standalone mode.

If you specify flm_options (not nullish), then the arguments fieldschema and object_schema are not used. If you don’t specify flm_options (nullish), the fieldschema option is used to parse the FLM options. If flm_options and fieldschema are both not specified (nullish) then we find the corresponding field schema in object_schema using the fieldname argument.

Note: the API guarantees that compile_flm(), compile_flm_fragment(), and compile_object() also work if no zoodb is set, and can be used w/o zoodb if you want to compile a single ad hoc object.

FLMSimpleContentCompiler.compile_flm_fragment(flm_content)

Compile the given string flm_content into an FLM fragment using the current environment. Will also apply any scanner, unless skip_scanner is true. Errors will be reported as per the config’s error policy.

  • flm_options is an object with keys ‘standalone’ (defaults to false) and is_block_level (defaults to null to let FLMFragment’s parser decide).

  • resource_info is attached to the fragment. If null, a ZooFLMResource info instance with null entries is created.

  • what is a description of what is being compiled. Useful for error messages.

  • skip_scanner defaults to false, so that the fragment scanner defined in the configuration of this processor is applied to the fragment by default. If this argument is set to a truthy value, then the fragment scanner is not applied.

  • fieldname can be set to the name of the object’s field name that is being compiled. It is used in error information reporting only.

Note: the API guarantees that compile_flm(), compile_flm_fragment(), and compile_object() also work if no zoodb is set, and can be used w/o zoodb if you want to compile a single ad hoc object.

FLMSimpleContentCompiler.compile_object(object_type, objid, obj, schema)

Compile all FLM-enabled fields of the database object obj in place, replacing each field’s string value with the compiled FLM fragment.

The list of compiled field names is stored in obj._zoodb.flm_fields so that later stages (e.g. process_data_dump()) know which fields to serialise back to text.

Note: compile_flm(), compile_flm_fragment(), and compile_object() also work if no zoodb is attached; they can be used to compile a stand-alone ad hoc object.

Arguments:

  • object_type (string) – The type name of the object being compiled.

  • objid (string) – The ID of the object being compiled.

  • obj (Object) – The database object whose FLM fields are compiled in place.

  • schema (Object) – The JSON Schema for object_type, used to discover which fields carry FLM content.

FLMSimpleContentCompiler.process_data_dump(data, options)

Prepare a data dump by converting in-memory FLM fragment objects back to a serialisable form. Exactly one of the following options must be chosen (defaulting to flm_fragments_to_flm_text when none is provided):

  • flm_fragments_keep_instances — keep the live FLM fragment objects in the dump as-is (useful for in-process consumers that can operate directly on fragments).

  • flm_fragments_to_flm_text — replace each compiled fragment with its original FLM source string. This is the default when neither of the other options is set.

  • flm_fragments_to_flm_dump — replace each compiled fragment with a key string of the form "flm.<n>" and collect all fragment data under data.flm_fragment_data (produced by FLMDataDumper). Useful for transferring pre-parsed fragment state across a network boundary.

The additional option flm_keep_zoodb_info_flm_fields (default true) controls whether obj._zoodb.flm_fields is retained in the dump. Set it to false to strip that internal bookkeeping field.

Arguments:

  • data (Object) – The data object being assembled for the dump.

  • options (Object) – Dump options; see above.

Returns:

Promise.<Object> – The (mutated) data object.