Overview

Packages

  • LegiScan
    • API
    • Cache
    • Common
    • Exceptions
    • Utility
  • PHP

Classes

  • LegiScan
  • LegiScan_CommandLine
  • LegiScan_Endpoint
  • LegiScan_Import
  • LegiScan_WebTest
  • LegiScan_Worker
  • Overview
  • Package
  • Class

Class LegiScan_Import

A bulk loader for JSON datasets and snapshots

Starting here is STRONGLY encouraged as a few hundred files are the equivalent of approximately 2 million individual API calls.

Using the getDataset API hooks, a manually downloaded copy of a weekly Public Dataset, or a custom subscriber onboarding snapshot, this script will extract the contents of the archives and import/update data as needed.

When using getDataset, the script can operate in two modes, --scan and --bulk, both of which import new and updated datasets.

The --scan mode acts like a interactive search / browser with command line filters to import eligible datasets. While in --bulk mode, the states[] and years[] variables in config.php will be used to select which datasets will be synchronized. The latter of which is most appropriate for scheduling for automating updates.

Show the command help

php legiscan-bulk.php --help

Use config.php settings to import datasets and answer yes to any prompts.

php legiscan-bulk.php --bulk --import --yes

Show a listing of all available datasets in California

php legiscan-bulk.php --scan --state CA

Import all new/changed datasets in 2020 with verbose output

php legiscan-bulk.php --scan --year 2020 --import --verbose

Import all new/changed 2016 special sessions in North Carolina

php legiscan-bulk.php --scan --state NC --special --year 2016 --import

Process example.zip but do not import, only show what would have been done

php legiscan-bulk.php --file example.zip --dry-run --debug

NOTE: This will not pull local copies of documents unless they are included in the archive, though appropriate stub records will be created.

Package: LegiScan\Utility
Copyright: 2010-2020 LegiScan LLC
License: https://opensource.org/licenses/BSD-2-Clause
Author: LegiScan API Team api@legiscan.com
See: LegiScan_Process
See: LegiScan_Bulk
Link: https://api.legiscan.com/dl/
Link: https://legiscan.com/datasets
Located at legiscan-bulk.php
Methods summary
public
# __construct( )

Class constructor, initilizes necessary API instances

Class constructor, initilizes necessary API instances

public
# updateDatasets( )

Use the states[] and years[] settings in config.php to find eligible datasets from LegiScan_Pull::getDatasetList to pull and import to the local database

Use the states[] and years[] settings in config.php to find eligible datasets from LegiScan_Pull::getDatasetList to pull and import to the local database

See

LegiScan_Pull
public
# scanDatasets( )

Use LegiScan_Pull::getDatasetList to generate listings of available datasets with optional filtering and import new or updated archives

Use LegiScan_Pull::getDatasetList to generate listings of available datasets with optional filtering and import new or updated archives

See

LegiScan_Pull
public
# processFile( )

Process a single LegiScan session dataset or subscriber onboarding ZIP archive that was previously manually downloaded from the public or private site with LegiScan_Bulk::importDataset

Process a single LegiScan session dataset or subscriber onboarding ZIP archive that was previously manually downloaded from the public or private site with LegiScan_Bulk::importDataset

See

LegiScan_Bulk

Link

https://legiscan.com/datasets
private
# processDownloads( mixed[] $downloads )

Take array of eligible dataset objects from LegiScan_Import::scanDatasets() or LegiScan_Import::updateDatasets() and download the getDataset payloads, extract the ZIP file and import with LegiScan_Bulk::importDataset

Take array of eligible dataset objects from LegiScan_Import::scanDatasets() or LegiScan_Import::updateDatasets() and download the getDataset payloads, extract the ZIP file and import with LegiScan_Bulk::importDataset

Parameters

$downloads
An array of dataset objects from getDatasetList to download and import

Throws

APIException
Properties summary
protected LegiScan_Pull $legiscan

Instance of LegiScan_Pull

Instance of LegiScan_Pull

#
protected LegiScan_Process $logic

Instance of LegiScan_Process

Instance of LegiScan_Process

#
protected LegiScan_Bulk $bulk

Instance of LegiScan_Bulk

Instance of LegiScan_Bulk

#
protected PDO $db

Database object PDO

Database object PDO

#
LegiScan API Client API documentation generated by ApiGen