Options¶
Every optional parameter across the main functions, grouped by what it affects. The defaults, the functions that accept each option, and the descriptions are read from polars-genson itself when these docs are built, so they match the released package.
Function key: ij infer_json_schema, ip infer_polars_schema, nj
normalise_json (the df.genson methods), ifp infer_from_parquet, nfp
normalise_from_parquet.
Input framing¶
| Option | Default | Functions | Description |
|---|---|---|---|
ndjson |
False |
ij, ip, nj, ifp, nfp | Whether to treat input as newline-delimited JSON |
ignore_outer_array |
True |
ij, ip, nj, ifp, nfp | Whether to treat top-level arrays as streams of objects |
wrap_root |
None |
ij, ip, nj, ifp, nfp | If a string, wrap each JSON row under that key before inference. |
Maps and records¶
See Maps and records.
| Option | Default | Functions | Description |
|---|---|---|---|
map_threshold |
20 |
ij, ip, nj, ifp, nfp | Number of keys above which a heterogeneous object may be rewritten as a map (unless overridden). |
map_max_required_keys |
None |
ij, ip, nj, ifp, nfp | Maximum number of required keys allowed for Map inference. |
force_field_types |
None |
ij, ip, nj, ifp, nfp | Explicit overrides for specific fields. |
force_parent_field_types |
None |
ij, ip, nj, ifp, nfp | Explicit overrides for fields based on their parent field name. |
unify_maps |
False |
ij, ip, nj, ifp, nfp | Enable unification of compatible but non-homogeneous record schemas into maps. |
no_unify |
None |
ij, ip, nj, ifp, nfp | Prevent unification of keys under these field names with their sibling record fields. |
no_root_map |
True |
ij, ip, nj, ifp, nfp | Prevent document root from becoming a map type, even if it meets map inference criteria |
Mixed types¶
See Mixed types.
| Option | Default | Functions | Description |
|---|---|---|---|
wrap_scalars |
True |
ij, ip, nj, ifp, nfp | Whether to promote scalar values into singleton objects when they appear in contexts where other rows provide objects. |
force_scalar_promotion |
None |
ij, ip, nj, ifp, nfp | Set of field names that should always be promoted to wrapped scalars, even when they appear as simple scalars. |
coerce_strings |
False |
nj, nfp | If True, attempt to parse numeric/boolean values from strings (e.g. "42" → 42, "true" → true). |
Normalised output¶
| Option | Default | Functions | Description |
|---|---|---|---|
empty_as_null |
True |
nj, nfp | If True, normalise empty arrays and empty maps to null. |
map_encoding |
'kv' |
nj, nfp | Encoding to use for Avro maps: |
decode |
True |
nj | Controls how the normalised JSON strings are decoded after normalisation: |
unnest |
True |
nj | Only applies if decode=True. |
Schema output¶
See Infer a schema.
| Option | Default | Functions | Description |
|---|---|---|---|
avro |
False (ij)True (ip)False (ifp) |
ij, ip, ifp | Whether to read the input as an Avro schema instead of JSON schema. |
merge_schemas |
True |
ij, ip | Whether to merge schemas from all rows (True) or return individual schemas (False) |
schema_uri |
'http://json-schema.org/schema#' |
ij, ifp | Schema URI to use for the generated schema |
Parquet output¶
See Normalise JSON stored in Parquet.
| Option | Default | Functions | Description |
|---|---|---|---|
output_path |
None |
ifp | Path to write schema JSON. |
output_column |
None |
nfp | Name for output column. |
typed |
False |
nfp | If True, write the normalised rows as a typed struct column (the dtype avro_to_polars_schema gives for the inferred schema) instead of JSON strings, so no str.json_decode is needed after reading it back. |
keep_columns |
None |
nfp | Input columns (e.g. an id) to copy unchanged into the output file, before the normalised column. |
extract_invariants |
None |
nfp | Fields that are invariant per key, each mapped to its determinant: the sibling field whose value determines it, e.g. {"labels": "id"} when every object with the same id carries the same labels. |
lookup_output_path |
None |
nfp | Parquet file for the lookup table, with string columns field, key and value (the subtree as JSON), in first-seen order. |
Performance and diagnostics¶
See Process large inputs.
| Option | Default | Functions | Description |
|---|---|---|---|
max_builders |
None |
ij, ip, nj, ifp, nfp | Maximum number of schema builders to create in parallel at once. |
profile |
False |
ij, ip, nj, ifp, nfp | Whether to print profiling information |
debug |
False |
ij, ip, ifp, nfp | Whether to print debug information |
verbosity |
'Normal' |
ij, ip, ifp | Whether to print verbose debug information |