Getting started

In this tutorial you will build a small researcher profile by hand, load it with the SDK, explore its contents in Python, and build a semantic index over it. By the end you will have a working, conforming profile directory and will have touched the main entry points of the package.

This tutorial assumes you can already install Python packages with pip and know the basics of running Python and editing YAML and Markdown files. You do not need to know anything about the profile format in advance — you will build one from scratch.

Learning objectives

• Create a conforming profile directory from plain YAML and Markdown files
• Load the profile with ResearcherProfile.from_files
• Read metadata, papers, and summaries in Python
• Build a per-profile embedding index and run a semantic search

Install the package

The embedding index needs the [embeddings] extra. Install it now:

pip install "researcher-profiles[embeddings]"

Step 1: Create the profile directory

A profile is a directory named after a slug (a lowercase, hyphenated identifier). Create one for a synthetic researcher named Jane Doe:

mkdir -p jane-doe/personality jane-doe/sources/summaries
cd jane-doe

Step 2: Write the metadata

Create profile.yaml. This is the structured metadata for the researcher:

name: Jane Doe
orcid: 0000-0002-1825-0097
affiliation: Example University
field: Computational Biology
subfields:
  - epigenomics
  - chromatin
summary: Jane Doe develops methods for region-set analysis.
expertise:
  - region-set-analysis
  - reproducible-workflows

Create config.json. The schema_version: 2 field marks this as a current-format profile:

{
  "schema_version": 2,
  "name": "Jane Doe",
  "orcid": "0000-0002-1825-0097",
  "slug": "jane-doe"
}

Step 3: Write the personality documents

Create personality/expertise.md. Sections describe what the researcher works on and cite papers by their paper_id in square brackets:

# Expertise

## Region set analysis
Jane developed methods for enrichment analysis of genomic region sets [doe2016example].

Create personality/SOUL.md, a short narrative of how the researcher thinks:

# How Jane thinks
Jane sees analysis problems as metadata problems.

Step 4: Add papers and a summary

Create sources/papers.yaml. Note the top-level papers: key — a bare list is rejected by the loader:

papers:
  - paper_id: doe2016example
    title: An example method for region set analysis
    year: 2016
    journal: Bioinformatics
    first_author: Doe
    status: downloaded

Create one summary file at sources/summaries/doe2016example.summary.md. The filename stem must match the paper’s paper_id:

Jane developed an example method for enrichment analysis of genomic region sets.

Your directory now looks like this:

jane-doe/
├── config.json
├── profile.yaml
├── personality/
│   ├── expertise.md
│   └── SOUL.md
└── sources/
    ├── papers.yaml
    └── summaries/
        └── doe2016example.summary.md

Step 5: Load the profile

From the parent directory, load the profile in Python:

from researcher_profiles import ResearcherProfile

p = ResearcherProfile.from_files("jane-doe")
print(p)

Expected output:

ResearcherProfile(slug='jane-doe', papers=?)

The papers=? marker means papers have not been read yet — the profile loads files lazily, only when you access them.

Step 6: Explore the profile

Read the metadata and corpus. Each access reads and caches the underlying file:

print(p.name)                       # Jane Doe
print(p.orcid)                      # 0000-0002-1825-0097
print(p.affiliation)                # Example University
print(p.field)                      # Computational Biology
print(p.metadata.subfields)         # ['epigenomics', 'chromatin']
print(len(p.papers))                # 1
print(p.papers[0].title)            # An example method for region set analysis
print(list(p.summaries.keys()))     # ['doe2016example']
print(p.config.schema_version)      # 2

Serialize the whole profile to a JSON-ready dict:

d = p.to_dict()
print(sorted(d.keys()))

Expected output:

['config', 'expertise', 'metadata', 'papers', 'path', 'slug', 'soul', 'summary_ids']

Every profile also has a level — lite, full, or deep — that records how deeply it was built; when unset it defaults to full, so this profile is full. Because Jane Doe has expertise.md and SOUL.md, she is persona-ready (p.has_persona is True); only full/deep profiles are. See profile depth levels.

Step 7: Build a semantic index

The embedding index lives inside the profile at meta/embeddings.sqlite, so the profile stays self-contained. Importing the embeddings subpackage attaches .build_index() and .search() to the profile:

import researcher_profiles.embeddings  # attaches .build_index / .search

report = p.build_index()
print(report.added, report.backend_name)

Expected output (the first run downloads the sentence-transformer model):

4 st:all-MiniLM-L6-v2

The count is the number of chunks indexed: the expertise sections, the SOUL document, and each paper summary are chunked and embedded separately.

Step 8: Search the corpus

Query the index. Hits come back ranked by cosine similarity:

hits = p.search("region set enrichment", k=3)
for h in hits:
    print(round(h.score, 3), h.source_type, h.source_id)

Expected output:

0.595 expertise expertise
0.568 paper_summary doe2016example
0.344 soul soul

The top hit is the expertise section that mentions region-set enrichment, then the paper summary, then the SOUL document.

Challenge

Add a second paper to sources/papers.yaml (give it a new paper_id, title, and year) and a matching sources/summaries/<paper_id>.summary.md. Rebuild the index and search again. How does the new summary rank for a query about your new paper’s topic?

Tip

After editing files on disk, call p.build_index(force=True) to drop and rebuild the index, or load a fresh ResearcherProfile — the in-memory profile caches file contents from the first access.

Summary

Summary

• A profile is a directory keyed by a slug, holding profile.yaml, config.json, personality/ Markdown, and sources/ papers and summaries
• ResearcherProfile.from_files(path) loads a profile and reads its files lazily on first access
• config.json carries schema_version: 2 to mark the current format
• The [embeddings] extra attaches build_index() and search(); the index is stored inside the profile at meta/embeddings.sqlite

Next steps

• The profile format — the full on-disk specification
• Build and search the embeddings index — indexing in more depth
• Python API reference — every class and method