Class LegiScan_Import

A bulk loader for JSON datasets and snapshots

Starting here is STRONGLY encouraged as a few hundred files are the equivalent of approximately 2 million individual API calls.

Using the getDataset API hooks, a manually downloaded copy of a weekly Public Dataset, or a custom subscriber onboarding snapshot, this script will extract the contents of the archives and import/update data as needed.

When using getDataset, the script can operate in two modes, --scan and --bulk, both of which import new and updated datasets.

The --scan mode acts like a interactive search / browser with command line filters to import eligible datasets. While in --bulk mode, the states[] and years[] variables in config.php will be used to select which datasets will be synchronized. The latter of which is most appropriate for scheduling for automating updates.

Show the command help

php legiscan-bulk.php --help

Use config.php settings to import datasets and answer yes to any prompts.

php legiscan-bulk.php --bulk --import --yes

Show a listing of all available datasets in California

php legiscan-bulk.php --scan --state CA

Import all new/changed datasets in 2020 with verbose output

php legiscan-bulk.php --scan --year 2020 --import --verbose

Import all new/changed 2016 special sessions in North Carolina

php legiscan-bulk.php --scan --state NC --special --year 2016 --import

Process example.zip but do not import, only show what would have been done

php legiscan-bulk.php --file example.zip --dry-run --debug

NOTE: This will not pull local copies of documents unless they are included in the archive, though appropriate stub records will be created.