Container based deployments
The simplest way to deploy Transiter is to use containers along with the included configurations for either Kubernetes (using the Transiter Helm Chart) or Docker compose.
During Transiter's continuous integration process, a
is built and uploaded to Docker hub.
<version> can be any of the following:
For stable releases, this is the version number of the release; for example
latestwill spin up the most recent stable release.
dev-latestwill spin up the most recent green build on master.
For production settings, pinning to a fixed stable release is, of course, recommended.
dev-latest versions may introduce breaking API changes at any time.
Kubernetes: the Transiter Helm Chart
The Transiter Helm Chart enables easy installation of Transiter on a Kubernetes cluster.
The Helm chart can be installed by referencing the built
helm install https://github.com/jamespfennell/transiter/raw/master/docker/helmchart-0.1.0.tgz
Or, by checking out the Git repository and installing from the file system:
helm install ./docker/helmchart
The Helm chart contains many configuration options. These are defined and described in the Helm Chart's values file. Some particularly important options are as follows.
versionis, by default,
latest. As per the above, in production this should usually be overridden with a fixed version.
If you are using custom feed parsers distributed as Python packages, these packages can be made available to Transiter using the
By default the Postgres container does not use a persistent volume. In production a persistent volume should be used so that data lives beyond the lifecycle of the container. This is configured in the
There is a Docker compose configuration in the Github repository that may be used a basis for a deployment. Using Docker compose has its limitations in terms of customization. The configuration file is mostly maintained to enable running a simpler CI pipeline on Travis, rather than being a recommended deployment mode.
When launched, the Docker compose config starts a Transiter instance listening on port 8000.
In production, you should use a volume for the Postgres data directory to ensure that data persists if the Postgres container is reset. By default that Postgres data directory does not do this. To enable a volume, change
and configure the volume
Interlude: Transiter service topology
Transiter uses five Docker containers to run, as well as an sixth init container:
The web service, which is a Gunicorn process running inside of the Transiter Docker image. This can be replicated to support more HTTP traffic.
The scheduler, which is also a Gunicorn process running inside of the Transiter Docker image. The scheduler can not be replicated, as this will result in duplicated updates being executed.
The executor, which is a Celery process running inside of the Transiter Docker image. This can be replicated to support more feed update processing throughput.
A Postgres instance, using the vanilla Postgres Docker image. This is where all the data lives.
A RabbitMQ instance, for the Celery cluster. This uses the vanilla rabbitmq Docker image.
The init container, which is based on the the Transiter Docker image. This container just initializes the database schema and then exits.
The topology is identical for non-Docker deployments, described below.
Running Transiter without containers
Transiter can be run on "bare metal" and in development often is. You system needs to have the following things installed: Python 3.6+, Postgres and RabbitMQ.
Setting up Postgres
Postgres is the only officially supported database for Transiter right now. Other systems such as SQLite and MySQL can be used with some tinkering and may be supported in the future; start an issue on the issue tracker if you're interested.
By default, Transiter connects to Postgres using the following configuration.
|Setting||Default||Environment variable to change the default|
In order to change the default, set the environment variable in any environment that a Transiter process (web service or task server) will be running.
After setting up Postgres and a database, you need to
initialize the database with the Transiter schema.
Assuming you're in a (virtual) environment in which the
Python package has been installed, this can be done by running:
transiterclt db reset
Running the web service
The Transiter web service response to HTTP requests. Transiter was built using the Flask framework and thus contains a WSGI app for launching a HTTP service.
Assuming you have the
transiter Python package installed,
Transiter's WSGI app is located at
So, for example, to launch a Transiter web service using Gunicorn:
If you're developing Transiter it can be helpful to launch the web service in debug mode so that it automatically restarts when you've made changes. Assuming you're in a Python (virtual) environment in which Transiter has been installed, run:
transiterclt launch webservice
This launches the Flask debugging server.
Running the scheduler
The task server is responsible for scheduler periodic feed updates. It is launched using:
transiterclt launch scheduler
The scheduler contains an HTTP interface and by default listens on localhost's port 5000.
The host and port can be changed using the environment variables
It is critical that these variables are also set for the web service and executor; otherwise,
the web service will be unable to contact the scheduler server which can result
in feeds not being auto updated after a transit system install.
Running the executor
transiterclt launch executor --logging-level info