Skip to content

End-to-End with run_together (single sample)

run_together runs a complete CartoScope pipeline with a single command — ingest → FICTURE → cell decode → asset packaging → image import → (optional) publish. It builds one Makefile and runs it, so the pipeline is resumable and parallel out of the box.

This tutorial mirrors the step-by-step Xenium tutorial but replaces the whole manual chain with one command, using the same public 10x Xenium human lung cancer dataset.

Which entry point should I use?

run_together is one of several ways to drive CartLoader. It shines for multi-platform and multi-sample runs and for one-command convenience. The per-platform orchestrators run_xenium / run_visiumhd and the individual modules remain fully supported. See Which interface should I use?.

Prepare input

Download and unpack the dataset exactly as in the step-by-step Xenium tutorial → Prepare Input. After unzipping, ${work_dir}/raw is the Xenium Ranger output directory.

The 10x_xenium profile reads this standard layout automatically — you do not list these files yourself:

1
2
3
4
5
6
${work_dir}/raw/
├── transcripts.parquet                 # or transcripts.csv.gz
├── cell_boundaries.csv.gz              # cell decode
├── cells.csv.gz                        # cell centroids (x_centroid, y_centroid)
├── analysis/clustering/gene_expression_graphclust/clusters.csv   # → "xeniumranger" prefix
└── morphology_focus/morphology_focus_000{0,1,2,3}.ome.tif        # dapi/boundary/rna/protein

See the full Xenium platform page for every path the profile looks for.

Define ID and parameters

1
2
3
4
DATA_ID="xenium-v1-humanlung-cancer-ffpe"   # dataset name; also names the output folder
train_width=18                              # FICTURE hexagon width (µm)
n_factor=24                                 # number of factors (comma-separated for several)
n_jobs=10

External tools on PATH

run_together delegates to CartLoader modules, which expect spatula, punkst (FICTURE2), tippecanoe, go-pmtiles, gdal, and pigz available (on PATH, or as built repo submodules for spatula/punkst). Unlike run_xenium, run_together does not take per-tool path flags.

Run the pipeline

Set up the environment 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# ====
# Replace each placeholder with the actual path on your system.  
# ====

work_dir=/path/to/work/directory        # path to work directory that contains the downloaded input data
cd $work_dir

# Define paths to required binaries and resources
spatula=/path/to/spatula/binary         # path to spatula executable
punkst=/path/to/punkst/binary           # path to FICTURE2 (punkst) executable
tippecanoe=/path/to/tippecanoe/binary   # path to tippecanoe executable
pmtiles=/path/to/pmtiles/binary         # path to pmtiles executable
aws=/path/to/aws/cli/binary             # path to AWS CLI binary

# (Optional) Define path to color map. 
cmap=/path/to/color/map                 # Path to fixed color map. `CartLoader` includes one at cartloader/assets/fixed_color_map_256.tsv.

# Number of jobs
n_jobs=10                               # If not specified, the number of jobs defaults to 1.

# Activate the bioconda environment
conda activate ENV_NAME                 # replace ENV_NAME with your conda environment name

Command 

1
2
3
4
5
6
7
cartloader run_together \
    --platform 10x_xenium \
    --in-dir  ${work_dir}/raw \
    --out-dir ${work_dir}/output/${DATA_ID} \
    --width ${train_width} \
    --n-factor ${n_factor} \
    -j ${n_jobs} --threads ${n_jobs}

Set up the environment

Fixed paths in the Docker Image

Tools and dependencies have fixed paths in the Docker image (for example, /usr/local/bin/pmtiles).

DO NOT modify paths of tools and dependencies manually.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
# ====
# Replace user-specific placeholders with actual paths on your system.
# ====
work_dir=/path/to/work/directory                        # path to work directory that contains the downloaded input data
cd $work_dir

# The following paths are fixed inside Docker. Do not modify them.
spatula=/app/cartloader/submodules/spatula/bin/spatula  # path to spatula executable
punkst=/app/cartloader/submodules/punkst                # path to FICTURE2 (punkst) executable
tippecanoe=/usr/local/bin/tippecanoe                    # path to tippecanoe executable
pmtiles=/usr/local/bin/pmtiles                          # path to pmtiles executable
aws=/usr/local/bin/aws                                  # path to AWS CLI binary

# (Optional) Define path to color map. 
cmap=/app/cartloader/assets/fixed_color_map_256.tsv     # Path to fixed color map. `CartLoader` includes one at cartloader/assets/fixed_color_map_256.tsv.

# Number of jobs
n_jobs=10                                               # If not specified, the number of jobs defaults to 1.

# Docker tag 
docker_tag=20260306

Command 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
docker run -it --rm \
    -v ${work_dir}:/data \
    weiqiuc/cartloader:${docker_tag} \
    run_together \
    --platform 10x_xenium \
    --in-dir  /data/raw \
    --out-dir /data/output/${DATA_ID} \
    --width ${train_width} \
    --n-factor ${n_factor} \
    -j ${n_jobs} --threads ${n_jobs}

Preview before running

Add --dry-run to write the Makefile and print every command (make -n) without executing. Inspect ${work_dir}/output/${DATA_ID}/run_together.mk and run_together.resolved.json to see exactly what will run.

Projection-only mode

To reuse models from a previous FICTURE run instead of training new ones, point --project-models at that run's FICTURE directory. run_together reads its ficture.params.json and re-projects every model it lists — no LDA training runs.

1
2
3
cartloader run_together --platform 10x_xenium \
    --in-dir ${work_dir}/raw --out-dir ${work_dir}/output/${DATA_ID} \
    --project-models /path/to/previous/fic --width 12 -j ${n_jobs}

Output

1
2
3
4
5
6
7
${work_dir}/output/${DATA_ID}/
├── run_together.mk               # the generated pipeline (Makefile)
├── run_together.resolved.json    # fully-resolved settings (provenance)
├── mk/                           # per-stage flag files that drive make
├── tsv/<id>/transcripts.unsorted.tsv.gz
├── fic/                          # FICTURE results (per sample under fic/samples/<id>/)
└── cartl/<id>/                   # packaged PMTiles + catalog.yaml  ← deploy this

The cartl/<id>/catalog.yaml plus its PMTiles is the deployable CartoScope asset. (A joint multi-sample run instead produces cartl/<multi_id>-<sample_id>/ per sample plus a cartl/multi-catalog.yaml — see the multi-sample tutorial.) See the per-module output details in sge_convert, run_ficture2, and run_cartload2.

Resume, re-run, and publish

  • Resume after a failure: just re-run the same command (or make -f .../run_together.mk -j N). Completed stages are skipped via their flag files.
  • Run part of the pipeline: --only ingest,ficture or --skip images. Excluded upstream stages are assumed already done.
  • Force a clean rebuild: --restart (make -B).
  • Publish (opt-in): annotation and S3 upload are separate CLI flags — --anno (with --tissue/--organism) and --s3-upload (with --collection). See the reference → Publish.

Next steps