Research Methodology
Key Insights
- How AcaciaFund transforms raw signals into quality-scored research — SQI, Bloom taxonomy, pipeline stages.
How AcaciaFund transforms raw signals into structured, quality-scored research articles.
Pipeline Stages
1. Source Ingestion
Content is drawn from three primary feeds:
- HackerNews API (news.ycombinator.com) — technology, business, and science stories ranked by community engagement (points, comments). Top ~30 stories per day are analyzed.
- arXiv API (arxiv.org) — academic preprints across computer science, physics, mathematics, quantitative biology, and finance. Filtered by relevance to pillar topics.
- PubMed — biomedical literature (archived — science pillar discontinued).
2. Entity Extraction & Summarization
Each source is processed through NLP pipelines for:
- Named Entity Recognition (NER) — extracting organizations, people, technologies, regulations, and financial instruments.
- Automatic Summarization — extractive summarization identifying key claims, numerical data, and conclusions.
- Topic Modeling — LDA-based classification into pillar-relevant topics and subtopics.
3. Bloom Taxonomy Classification
Every article is assessed across six cognitive levels:
| Level | Description | Example Question |
|---|---|---|
| Remember | Recall facts and basic concepts | Which pillar does this article belong to? |
| Understand | Explain ideas and concepts | What is the primary domain of this article? |
| Apply | Use information in new situations | How can these findings be applied in practice? |
| Analyze | Draw connections among ideas | What assumptions underlie this analysis? |
| Evaluate | Justify a stand or decision | How strong is the evidence presented? |
| Create | Produce new or original work | Design an experiment to test this hypothesis. |
4. Signal Quality Index (SQI) Computation
SQI is a composable metric computed per article as a weighted combination:
SQI = w1 · Authority + w2 · Freshness + w3 · Consensus + w4 · Relevance where: Authority = domain reputation score (0–1) Freshness = 1 - (days_since_publication / freshness_window) Consensus = cross-source agreement rate Relevance = keyword overlap with pillar taxonomy Default weights: w1=0.35, w2=0.25, w3=0.25, w4=0.15
Weights are adjustable per pillar to prioritize different quality dimensions (e.g., AML favors authority, Data Engineering favors freshness).
5. Cross-Pillar Analysis
Each article is analyzed for connections to other pillars via:
- Shared entity references — organizations, regulations, or technologies that appear across pillars
- Source overlap — articles from different pillars citing the same source
- Topic bridging — latent topic modeling revealing cross-domain themes
6. Quality Gates & Serving
Before publication, each article passes through quality gates:
- Schema validation — Pydantic ensures all required fields are present and correctly typed
- Source diversity check — minimum source count and domain diversity thresholds
- SQI threshold — minimum SQI of 0.35 for publication
- Deterministic build — same registry.json always produces identical output
The final artifact is static HTML deployed to Cloudflare Pages via python3.13 build.py.
Last updated: 2026-06-08
Article Metadata
Further Reading
Databricks Blog
Lakehouse, Spark, Delta Lake, Unity Catalog — engineering blog
Apache Kafka
Kafka documentation, KIPs, and ecosystem updates
Apache Flink
Flink documentation and release notes
Apache Iceberg
Iceberg table format — specs, REST catalog, performance
dbt Blog
dbt Labs engineering blog — analytics engineering, Semantic Layer
Dagster Blog
Dagster orchestration — software-defined assets, IO managers
Feynman Concept Cards
Master each concept: read the ELI5, explore analogies, work examples, and teach it back.
DataOps is a concept in best practices. In simple terms, DataOps covers best practices in Data Engineering. This data engineering concept addresses key topics in the best practices in data engineering domain. Also known as: DataOps practices, data operation
Analogy
Example
Find Gaps
Explain DataOps as if teaching a colleague who is new to best practices. Cover: what it is, how it works, and why it matters.
Create
Create a checklist that demonstrates DataOps in a real-world best practices scenario. Walk through your design decisions.
Show solution
A checklist for DataOps should include: 1. The core components of dataops 2. How they interact 3. Expected outcomes or outputs
Research is a concept in specialized. In simple terms, A concept related to research
Analogy
Example
Find Gaps
Explain Research as if teaching a colleague who is new to specialized. Cover: what it is, how it works, and why it matters.
Create
Create a diagram that demonstrates Research in a real-world specialized scenario. Walk through your design decisions.
Show solution
A diagram for Research should include: 1. The core components of research 2. How they interact 3. Expected outcomes or outputs