Class LegiScan_Import
A bulk loader for JSON datasets and snapshots
Starting here is STRONGLY encouraged as a few hundred files are the equivalent of approximately 2 million individual API calls.
Using the getDataset API hooks, a manually downloaded copy of a weekly Public Dataset, or a custom subscriber onboarding snapshot, this script will extract the contents of the archives and import/update data as needed.
When using getDataset, the script can operate in two modes, --scan
and --bulk
,
both of which import new and updated datasets.
The --scan
mode acts like a interactive search / browser with command line filters
to import eligible datasets. While in --bulk
mode, the states[]
and years[]
variables in config.php
will be used to select which datasets will be
synchronized. The latter of which is most appropriate for scheduling for automating
updates.
Show the command help
php legiscan-bulk.php --help
Use config.php
settings to import datasets and answer yes to any prompts.
php legiscan-bulk.php --bulk --import --yes
Show a listing of all available datasets in California
php legiscan-bulk.php --scan --state CA
Import all new/changed datasets in 2020 with verbose output
php legiscan-bulk.php --scan --year 2020 --import --verbose
Import all new/changed 2016 special sessions in North Carolina
php legiscan-bulk.php --scan --state NC --special --year 2016 --import
Process example.zip
but do not import, only show what would have been done
php legiscan-bulk.php --file example.zip --dry-run --debug
NOTE: This will not pull local copies of documents unless they are included in the archive, though appropriate stub records will be created.
Copyright: 2010-2020 LegiScan LLC
License: https://opensource.org/licenses/BSD-2-Clause
Author: LegiScan API Team api@legiscan.com
See:
LegiScan_Process
See:
LegiScan_Bulk
Link: https://api.legiscan.com/dl/
Link: https://legiscan.com/datasets
Located at legiscan-bulk.php
public
|
|
public
|
#
updateDatasets( )
Use the |
public
|
#
scanDatasets( )
Use LegiScan_Pull::getDatasetList to generate listings of available datasets with optional filtering and import new or updated archives |
public
|
#
processFile( )
Process a single LegiScan session dataset or subscriber onboarding ZIP archive that was previously manually downloaded from the public or private site with LegiScan_Bulk::importDataset |
private
|
#
processDownloads( mixed[] $downloads )
Take array of eligible |
protected
|
$legiscan
Instance of |
|
protected
|
$logic
Instance of |
|
protected
|
$bulk
Instance of |
|
protected
|
$db
Database object |