liberata_metrics.generators package

Overview

The generators module provides tools for creating synthetic academic knowledge graphs and capital matrices. This is essential for:

• Testing: Validate metrics computation without relying on real data

• Benchmarking: Compare algorithm performance on controlled datasets

• Experimentation: Explore “what-if” scenarios with specific properties

• Education: Understand how the Liberata system works with transparent synthetic data

Data Structures Generated

The generators create several interconnected data structures:

References Matrix

A sparse matrix encoding citation relationships between manuscripts.

• Shape: (num_manuscripts, num_manuscripts)

• Entries: Number of times manuscript i cites manuscript j

• Format: COO (coordinate format) for memory efficiency

• Sparsity: Controlled via citation_density parameter

Capital Matrix

Represents how manuscripts generate academic capital that contributors accrue.

• Shape: (num_manuscripts, num_contributors)

• Entries: Capital accrued by contributor j from manuscript i

• Format: COO sparse matrix

• Extended format: May include multiple capital blocks for different capital types

ID Mappings

Dictionaries mapping human-readable IDs to matrix indices:

• manuscript_id_to_index: Maps manuscript UUIDs to row indices

• contributor_id_to_index: Maps contributor UUIDs to column indices

• Used for linking computed metrics back to real entities

Metadata

Supplementary information about generated entities:

• Manuscript metadata: upload dates, topics, retraction status

• Contributor metadata: attributes and relationships

• Topic assignments based on OpenAlex topics

Key Functions

generate_references_matrix(num_manuscripts, citation_density=0.05, start_date=date(2020, 1, 1), end_date=date(2024, 1, 1), seed=None)

Generate a synthetic citation network.

Parameters:

• num_manuscripts – Number of manuscripts to generate

• citation_density – Sparsity of citation relationships (0-1)

• start_date – Earliest manuscript publication date

• end_date – Latest manuscript publication date

• seed – Random seed for reproducibility

Returns:

Tuple of (references_matrix, manuscript_ids, id_to_index_map, dates_map, metadata_df, capital_matrix, contributor_matrix)

Example:

references, ms_ids, ms_map, dates, meta, capital, contribs = \
    generate_references_matrix(num_manuscripts=500, citation_density=0.02)

Configuration

Generators are configured via YAML files. Example matrix_config.yaml:

# Number of manuscripts to generate
num_manuscripts: 1000

# Citation network sparsity (0 = no citations, 1 = complete graph)
citation_density: 0.05

# Time range for manuscript generation
start_date: 2020-01-01
end_date: 2024-12-31

# Random seed for reproducibility
seed: 42

# Use OpenAlex topics (True) or generic names (False)
use_openalex_topics: True

# Output format and location
output_dir: ./test_scripts/output
output_format: coo  # Sparse coordinate format

Submodules

• liberata_metrics.generators.generate_matrices module
  • Manuscript
  • build_COO()
  • build_capital_matrix()
  • generate_capital_time_series()
  • generate_references_matrix()
  • generate_shares_matrix()
  • get_capital_earlier()
  • get_references_earlier()
  • update_retractions_graph()

Common Usage Patterns

Generate test data for unit tests:

from liberata_metrics.generators import generate_references_matrix

# Create reproducible test data
refs, ms_ids, ms_map, dates, meta, cap, contribs = \
    generate_references_matrix(
        num_manuscripts=100,
        citation_density=0.03,
        seed=12345  # Fixed seed for reproducibility
    )

# Use in tests
assert refs.shape[0] == 100
assert len(ms_ids) == 100

Generate data with specific properties:

# Sparse citation network
sparse_refs, *_ = generate_references_matrix(
    num_manuscripts=1000,
    citation_density=0.001  # Very sparse
)

# Dense network for comparing algorithms
dense_refs, *_ = generate_references_matrix(
    num_manuscripts=1000,
    citation_density=0.1  # Much denser
)

Load from configuration file:

import yaml
from liberata_metrics.generators import generate_references_matrix

with open('config/matrix_config.yaml') as f:
    config = yaml.safe_load(f)

refs, *_ = generate_references_matrix(**config)

Module Contents

liberata_metrics.generators.build_capital_matrix(references: spmatrix, shares: spmatrix) → coo_matrix

computes capital matrix given shares and reference at current time

liberata_metrics.generators.generate_capital_time_series(references: spmatrix, shares: spmatrix, manuscript_index_map: Dict[str, int], upload_dates: Dict[str, date], start_date: date, end_date: date, time_step: timedelta) → Dict[str, object]

generates time series data for capital matrices at different time snapshots

liberata_metrics.generators.generate_references_matrix(num_manuscripts: int, citation_density: float = 0.05, start_date: date = datetime.date(2020, 1, 1), end_date: date = datetime.date(2024, 1, 1), seed: int | None = None) → Tuple[coo_matrix, List[str], Dict[str, int], Dict[str, date], DataFrame, coo_matrix, coo_matrix]

build toy references matrix

liberata_metrics.generators.generate_shares_matrix(manuscript_ids: Iterable[str], manuscript_index_map: Dict[str, int], num_contributors: int, avg_contributors_per_man: int, std_contributors_per_man: int, contributor_shares_dist: str = 'dirichlet', pareto_alpha: int = 2, seed: int | None = None) → Tuple[coo_matrix, List[str], Dict[str, int]]

build toy shares matrix

liberata_metrics.generators.get_capital_earlier(references: spmatrix, shares: spmatrix, manuscript_index_map: Dict[str, int], upload_dates: Dict[str, date], time_cutoff: date) → spmatrix

builds the earlier capital matrix corresponding to given time cutoff

liberata_metrics.generators.update_retractions_graph(references: spmatrix, retractions: spmatrix | None, manuscript_index_map: Dict[str, int], newly_retracted_manuscript_ids: List[str], manuscripts_metadata: DataFrame | list) → Tuple[csr_matrix, csr_matrix, DataFrame | list]

This function handles the movement of citation rows from the references graph to the retractions graph when manuscripts are newly retracted. It also updates previously retracted manuscripts to capture any new citations that occurred after retraction.

Parameters:

• references (sparse.spmatrix) – Sparse matrix where columns represent manuscripts and rows represent cited works. Non-zero entries represent citation relationships.

• retractions (Union[sparse.spmatrix, None]) – Sparse matrix of the same shape as references containing previously retracted manuscript citations. If None, an empty sparse matrix is initialized.

• manuscript_index_map (Dict[str, int]) – Mapping from manuscript IDs to their corresponding row indices in the matrices.

• newly_retracted_manuscript_ids (List[str]) – List of manuscript IDs that have been newly retracted in this update.

• manuscripts_metadata (Union[pd.DataFrame, list]) – Metadata associated with manuscripts. If a DataFrame, the retraction_cutoff column is updated for newly retracted manuscripts.

Returns:

A tuple containing: - retractions (sparse.coo_matrix): Updated retractions graph with newly retracted

and previously retracted manuscript citations.

• references (sparse.coo_matrix): Updated references graph with retracted manuscript rows zeroed out.

• manuscripts_metadata (Union[pd.DataFrame, list]): Updated metadata with retraction information for newly retracted manuscripts.

Return type:

Tuple[sparse.csr_matrix, sparse.csr_matrix, Union[pd.DataFrame, list]]

Notes

• Both newly retracted and previously retracted manuscript rows are removed from the references graph and consolidated in the retractions graph.

• If manuscripts_metadata is not a DataFrame, a warning is issued and metadata is returned unchanged.

• Output matrices are converted to COO format for sparse representation efficiency.