Command Line Reference

This page serves as a quick reference guide for working with Arches through a command prompt. Along with default Django commands, a good deal of Arches operations have been added to manage.py. In a command prompt, [activate your virtual environment](Dev-Installation#4-activate-the-virtual-environment), then run the following commands from your root app directory (the one that contains manage.py).

_All file or directory path parameters (-s, -c, -d) should be absolute paths._

Installation Commands

installing from a local repo clone

pip install -e .
-e This argument with the value . indicates to pip that it should link the local directory with the virtual environment.

Installs Arches into your virtual environment from a local clone of the archesproject/arches repo, or your own fork of that repo. To do this properly, create a new virtual environment and activate it, clone the repo you want, enter that repo’s root directory, and then run the command. Also, this command must be followed by:

pip install -r arches/install/requirements.txt

in order to properly install all of Arches’ python requirements. Make sure to use \ instead of / on Windows.

creating an Arches project

arches-project create <name_of_project> [{-d|--directory} <directory_name>]
-d, --directory
 (Optional) The name of the directory you’d like your new project located in.

creating (or recreating) the database

python manage.py packages -o setup_db
-o packages operation, in this case setup_db

Deletes and recreates the database, as defined by settings.DATABASES['default']. Likewise, this command will remove all existing data.

loading a package into a project

python manage.py packages -o load_package -s source_of_package [-db true]
-db Add this argument with the value true to force the destruction and recreation of your database before loading the package.

The source (-s) of a package can be either a path to a local directory, the location of a local zipfile containing a package, or the url to a github repo archive that contains a package. For example, loading the v4 sample package from where it resides in github would just be:

python manage.py packages -o load_package -s https://github.com/archesproject/arches4-example-pkg/archive/master.zip

ElasticSearch Management

reindex the database

python manage.py es index_database

Important

Memory usage will continuously increase during indexing, if you do not have DEBUG = False. This is because Django will store all db queries in memory, and a lot of them happen during indexing. Be wary of this when indexing a large database, or on servers with small memory provisions.

register a custom index

python manage.py es add_index --name {index name}

See Adding a Custom Index

Import Commands

Import Resource Models or Branches in archesjson format

python manage.py packages -o import_graphs [-s path_to_json_directory_or_file]
-s Path to the source file you are importing. If not specified, the command will look to settings.RESOURCE_GRAPH_LOCATIONS for directory paths

Import reference data in skos/rdf format

python manage.py packages -o import_reference_data -s 'path_to_rdf_file' [-ow {'overwrite'|'ignore'}] [-st {'stage'|'keep'}]

Import business data

python manage.py packages -o import_business_data -s 'path_to_source_file' [-c 'path_to_mapping_file'] [-ow '{overwrite'|'append'}] [--create_concepts {'create'|'append'}] [--bulk_load]
-c The path to the mapping file. The mapping file tells Arches how to map the columns from your csv file to the nodes in your resource graph. This option is required if there is not a mapping file named the same as the business data file and in the same directory with extension ‘.mapping’ instead of ‘.csv’ or ‘.json’.
-ow Determines how resources with duplicate ResourceIDs will be handled: append adds more tile data to an existing resource; overwrite replaces any existing resource with the imported data. This option only applies to CSV import. JSON import always overwrites.
-bulk, --bulk_load
 Bulk load values into the database. By setting this flag the system will use Django’s bulk_create operation. The model’s save() method will not be called, and the pre_save and post_save signals will not be sent.
--create_concepts
 Creates or appends concepts and collections to your rdm according to the option you select. create will create concepts and collections and associate them to the mapped nodes. append will append concepts to the existing collections assigned to the mapped nodes and create collections for nodes that do not have an assigned collection.

See Importing a CSV for CSV formatting requirements

Import resource to resource relations

python manage.py packages -o import_business_data_relations -s 'path_to_relations_file'

See Importing Resource Relations

Export Commands

export branch or resource model schema

python manage.py packages -o export_graphs -d 'path_to_destination_directory' -g uuid/branches/resource_models/all
-o packages operation, in this case export_graphs
-d Absolute path to destination directory
-g UUID of specific graph, or branches for all branches, resource_models for all resource models, or all for everything.

Exports Resource Models and/or Branches. Note that sometimes (as in this case) Resource Models and Branches are generically called “graphs”.

export business data to csv or json

python manage.py packages -o export_business_data -d 'path_to_destination_directory' -f 'csv' or 'json' [-c 'path_to_mapping_file' -g 'resource_model_uuid' -single_file]
-o packages operation, in this case export_business_data
-d Absolute path to destination directory
-f Export format, must be csv or json
-c (required for csv) Absolute path to the mapping file you would like to use for your csv export.
-single_file (optional for csv) Use this parameter if you’d like to export your grouped data to the same csv file as the rest of your data.
-g (required for json, optional for csv) The resource model UUID whose instances you would like to export.

Exports business data to csv or json depending on the -f parameter specified. For csv export a mapping file is required. The exporter will export all resources of the type indicated in the resource_model_id property of the mapping file and the -g parameter will be ignored. For json export no mapping file is required, instead a resource model uuid should be passed into the -g command.

Note that in a Windows command prompt, you may need to replace ' with ".

export business data to shapefile

python manage.py export shp -t 'name_of_db_view' -d 'output_directory'
-t A resource instance database view
-d The destination directory for point, line, and polygon shapefiles, created when the command is run.

business data export examples

python manage.py packages -o export_business_data -f 'csv' -c 'path_to_mapping_file'

Exports all business data of the resource model indicated in the mapping file. Two files are created. The first file contains one row per resource (if you resources all have the same geometry type this file can be used to create a shape file in QGIS or other program). The second file contains the grouped attributes of your resources (for instance, alternate names, additional classifications, etc.).

python manage.py packages -o export_business_data -f 'json' -g 'resource_model_id'
-f ‘json’ or ‘csv’

Exports all business data of the passed in resource_model_id to the specified file format. Take a look at the RESOURCE_FORMATERS dictionary in Arches’ settings.py for some other interesting options.

Other Data Management Commands

python manage.py resources remove_resources [-g graph_id][-y]
-g A Graph UUID to remove all the resource instances of.
-y Forces this command to run without interactive confirmation.

Removes all resources from your database, but leaves the all resources models, branches, thesauri, and collections intact.

python manage.py packages -o create_mapping_file -d 'path_to_destination_directory' -g 'comma separated graph uuids'
-d Path to directory to place the output in.
-g One or more graph UUIDs to create a mapping for.

This mimics the ‘Create Mapping File’ command from the Arches Designer UI.

python manage.py packages -o import_mapping_file -s 'path_to_mapping_file'

Imports a mapping file for a particular resource model. This will be used as the export mapping file for a resource by default (e.g. for search export).

Ontology Commands

load an ontology

python manage.py load_ontology [-s <path to ontology directory>]
-s Path to new ontology directory to load

Managing Functions, DataTypes, Widgets, and Card Components

To learn how to build new Functions, DataTypes, or Widgets, please see ref:Creating New Functions, Widgets, Card Components, and DataTypes. Note that when importing Widgets and associated DataTypes, Widgets must be registered first.

function commands

list registered functions

python manage.py fn list

Lists all currently registered functions.

registering functions

python manage.py fn register --source path/to/your/function.py

Register a newly created function. These .py files should sit in your projects functions directory.

unregistering functions

python manage.py fn unregister -n 'Sample Function'

Unregister a function. Use the function name that is returned by fn list.

datatype commands

list registered datatypes

python manage.py datatype list

Lists all currently registered datatypes.

registering and updating datatypes

python manage.py datatype register --source /Users/me/Documents/projects/mynewproject/mynewproject/datatypes/wkt_point.py

Registers a new datatype, in this example as defined in wkt_point.py.

python manage.py datatype update --source /Users/me/Documents/projects/mynewproject/mynewproject/datatypes/wkt_point.py

Updates a datatype, necessary anytime changes are made to your datatype’s properties.

-source Location of the .py file that defines the datatype.

unregister a datatype

python manage.py datatype unregister -d 'wkt-point'

Unregisters a datatype, in this example a datatype named wkt-point.

-d Name of datatype to unregister. Use the datatype name that is returned by datatype list.

widget commands

All widget-related commands are identical to those for datatypes, just substitute widget for datatype. Also note that where datatypes are defined in .py files, widgets are defined in .json files.

card component commands

All component-related commands are identical to those for widgets, just substitute card_component for widget. JSON files are used to register Card Components.

Creating Map Layers

See Creating New Map Layers for file format requirements and other in-depth information.

MapBox

python manage.py packages -o add_mapbox_layer -j /path/to/mapbox_style.json -n "New MapBox Layer" [{-b|--is_basemap}] [{-i|--layer_icon} 'icon_class'}]
-j The path to the Mapbox JSON file
-n The name of the Mapbox layer

Other Useful Django Commands

Run the django webserver

python manage.py runserver

Run the Django dev server. Add 0.0.0.0:8000 to explicitly set the host and port, which may be necessary when using remote servers, like an AWS EC2 instance. More about runserver.

collect static files

python manage.py collectstatic

Collects all static files and places them in a single directory. Generally only necessary in production. Also allows all static files to be hosted on another server).

Django’s full manage.py commands are documented here.