Human acceptance test plan
This handbook tests ACORN as a person would use it. It is written for a tester who is comfortable copying commands into PowerShell but does not need to know Rust, JSON schemas, databases, containers, or metadata standards.
The companion Gherkin behavior specification is stored at tests/acceptance/features/acorn_cli.feature. Scenario tags and IDs match this handbook. The feature file has no Cucumber step definitions; use this pageβs copy-paste commands. The intake section also supplies a runnable Python rehearsal for INTAKE-01β09, with separate visual, interactive, and live-service checks.
| Gherkin tag | Meaning |
|---|---|
@baseline | Run during ordinary acceptance testing |
@network | Requires internet access and a third-party service |
@interactive or @manual | Requires a person to observe or stop the command |
@owner | Requires adult/repository-owner supervision and authorization |
@gap | Confirms a currently incomplete interface fails clearly |
@optional | Depends on a build feature or installed program |
@demo | Required before accepting the intake demonstration for a release |
The baseline tests work on disposable copies under target/qa-acceptance/. Optional tests are clearly labeled when they require the internet, Chrome, Git, a container runtime, credentials, or an account-changing action.
What ACORN is for
ACORN manages research activity data (RAD) and the artifacts and workflows around it.
| Capability | Likely use case | Commands |
|---|---|---|
| Validate content | Find malformed metadata, broken links, difficult prose, and incomplete research records | check |
| Normalize content | Fix common values, formatting, contact data, vocabularies, and missing media references | format |
| Add linked data | Produce JSON-LD with standards-based semantic context | link |
| Publish artifacts | Create PDF, PowerPoint, Markdown, YAML, JSON, CFF, and BagIt output | export |
| Convert standards | Crosswalk DataCite, DCAT, Dublin Core, InvenioRDM, and HuWise metadata | export --from ... --to ... |
| Discover identifiers | Find DOI, ORCID, ROR, arXiv, ISBN, RAiD, Handle, and other identifiers in text and documents | gather |
| Resolve and catalog research | Look up identifier metadata and retain discovery/candidate history | gather --resolve |
| Move research content | Download remote buckets or import local/remote content and BagIt archives | download, import |
| Review form submissions | Stage collected JSONL, inspect changes, apply corrections, and graduate metadata | intake |
| Assemble sponsor highlights | Generate a combined local PDF from reviewed project logbooks | report --combine |
| Manage local models | Inspect GGUF metadata, download model weights, and configure inference applications | import model, download model, sync |
| Describe APIs | Turn an OpenAPI document into an ACORN endpoint resource | import spec |
| Identify software | Calculate and verify Software Hash Identifiers for files, directories, and Git objects | swhid |
| Diagnose a machine | Report system, memory, network, GPU, and software readiness | doctor |
| Support agents | Extract ACORNβs embedded agent skill | skill |
| Run interactive/services | Open the TUI or run/create GitLab bots and runners | tui, serve, create |
The CLI also supports JSON, JSONC, YAML, Markdown RAD, CFF, DataCite, DCAT, Dublin Core native JSON, JSON-LD, RDF/XML, Turtle, InvenioRDM, HuWise, RAiD, plain text, PDF, and Word document inputs where appropriate. Global controls cover offline operation, local SQLite or DuckDB storage, watch mode, concurrency, cache lifetime, verbosity, and quiet output.
Rules for the tester
- Open one PowerShell window at the repository root. Keep using that same window so the setup variables remain available.
- Run the setup once. Each later code block is one test step.
- A green
[PASS]is a pass. A red error, thrown[FAIL], missing artifact, crash, hang, or result that contradicts the stated expectation is a fail. - Warnings about writing quality are valid test results unless the step says there should be no warnings.
- Do not run an owner-supervised test without the repository ownerβs help. Those tests can change accounts, create containers, download large files, or expose a service.
- Record the ACORN version, Windows version, test ID, result, and the complete terminal output for every failure.
Suggested result record:
Test ID:
Result: PASS / FAIL / SKIP
ACORN version:
Computer/OS:
What happened:
Expected:
Screenshot or pasted output:
0. Setup
SETUP-01 β Build the exact checked-out source
Skip this step only when testing an installed release. A successful build ends with Finished.
cargo build --package acorn-cli --bin acorn
if ($LASTEXITCODE -ne 0) { throw "[FAIL] ACORN did not build" }
Write-Host "[PASS] ACORN built successfully" -ForegroundColor Green
SETUP-02 β Create a disposable test workspace
This copies fixtures rather than editing the originals. It also creates environment variables used below and isolates normal output beneath target/qa-acceptance.
For a source build:
. .\scripts\qa\Setup-AcornAcceptance.ps1 -AcornPath .\target\debug\acorn.exe
For an installed release, use this instead:
. .\scripts\qa\Setup-AcornAcceptance.ps1 -AcornPath (Get-Command acorn).Source
Expected: the script prints paths for the binary, inputs, outputs, database, and optional browser. Do not close this PowerShell window.
1. Installation and command discovery
CLI-01 β Verify the executable reports a version
$text = & $env:ACORN_BIN --version 2>&1
$code = $LASTEXITCODE
$text
Confirm-ExitCode -Actual $code -Expected 0 -Test "Version command"
if ($text -notmatch '^acorn-cli \d+\.\d+\.\d+') { throw "[FAIL] Version is not in the expected form" }
Expected: output similar to acorn-cli 0.1.76.
CLI-02 β Verify all supported top-level commands are advertised
$help = & $env:ACORN_BIN --help 2>&1 | Out-String
$code = $LASTEXITCODE
$required = 'check','create','doctor','download','export','format','gather','import','intake','link','report','swhid','skill','serve','sync','tui'
$missing = $required | Where-Object { $help -notmatch "(?m)^ $_\s" }
$help
Confirm-ExitCode -Actual $code -Expected 0 -Test "Top-level help"
if ($missing) { throw "[FAIL] Missing commands: $($missing -join ', ')" }
Write-Host "[PASS] All command families are advertised" -ForegroundColor Green
CLI-03 β Verify every command can explain itself
$commands = @(
@('check'), @('create'), @('create','runner'), @('create','bot'),
@('doctor'), @('download'), @('download','model'), @('export'),
@('format'), @('gather'), @('import'), @('import','model'),
@('import','spec'), @('intake'), @('intake','stage'), @('intake','inspect'),
@('intake','apply'), @('intake','run'), @('link'), @('report'), @('swhid'), @('skill'),
@('serve'), @('serve','bot'), @('sync'), @('tui')
)
foreach ($command in $commands) {
& $env:ACORN_BIN @command --help *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Help failed for: acorn $($command -join ' ')" }
}
Write-Host "[PASS] Every supported command returned help" -ForegroundColor Green
CLI-04 β Verify aliases work
& $env:ACORN_BIN --offline --no-local-database lint "$env:ACORN_QA_INPUT\valid-rad\index.json" --disable-website-checks --skip prose *> $null
$lintCode = $LASTEXITCODE
& $env:ACORN_BIN --no-local-database fmt "$env:ACORN_QA_INPUT\format-rad\index.json" --dry-run --no-color *> $null
$formatCode = $LASTEXITCODE
& $env:ACORN_BIN --offline --no-local-database harvest --text "doi:10.1038/nature12373" --raw *> $null
$gatherCode = $LASTEXITCODE
if (@($lintCode, $formatCode, $gatherCode) | Where-Object { $_ -ne 0 }) { throw "[FAIL] One or more aliases failed" }
Write-Host "[PASS] lint, fmt, and harvest aliases work" -ForegroundColor Green
2. Check research data and documents
These tests use --skip prose so the baseline does not need to download or synchronize the external prose analyzer. Website checks are disabled unless a test explicitly exercises the internet.
CHECK-01 β Check a valid RAD JSON file
& $env:ACORN_BIN --offline --no-local-database check "$env:ACORN_QA_INPUT\valid-rad\index.json" --disable-website-checks --skip prose --all
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Valid RAD JSON check"
Expected: a results table appears and the schema count is zero. A readability warning is acceptable.
CHECK-02 β Detect several errors in malformed RAD
$report = "$env:ACORN_QA_OUTPUT\malformed-check.txt"
& $env:ACORN_BIN --offline --no-local-database check "$env:ACORN_QA_INPUT\malformed-rad.json" --disable-website-checks --skip prose --no-fail *>&1 | Tee-Object -FilePath $report
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Malformed RAD is reported with --no-fail"
Confirm-Text -Path $report -Pattern "Deserialization error" -Test "Malformed fields were detected"
Expected: ACORN reports more than one deserialization or schema problem rather than stopping after the first one.
CHECK-03 β Check supported RAD encodings and metadata standards
$cases = @(
@('valid-rad\index.jsonc','rads'),
@('rad.yaml','rads'),
@('rad.md','rads'),
@('CITATION.cff','cff'),
@('raid.json','raid'),
@('datacite.json','datacite'),
@('dcat.json','dcat'),
@('invenio-single.json','invenio'),
@('huwise.json','huwise')
)
foreach ($case in $cases) {
$path = if ($case[0] -match '^(datacite|dcat|invenio|huwise)') { "$env:ACORN_QA_INPUT\schema\$($case[0])" } else { "$env:ACORN_QA_INPUT\$($case[0])" }
& $env:ACORN_BIN --offline --no-local-database check $path --standard $case[1] --disable-website-checks --skip prose --no-fail *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] $($case[1]) check failed" }
}
Write-Host "[PASS] JSONC, YAML, Markdown, CFF, RAiD, DataCite, DCAT, InvenioRDM, and HuWise inputs were accepted" -ForegroundColor Green
CHECK-03B β Check Dublin Core RDF syntaxes
$ttl = "$env:ACORN_QA_OUTPUT\dublin-core.ttl"
$jsonld = "$env:ACORN_QA_OUTPUT\dublin-core.jsonld"
$rdf = "$env:ACORN_QA_OUTPUT\dublin-core.rdf"
'@prefix dcterms: <http://purl.org/dc/terms/> . <https://example.org/acceptance> dcterms:title "Acceptance dataset"@en .' | Set-Content -LiteralPath $ttl
'[{"@id":"https://example.org/acceptance","http://purl.org/dc/terms/title":[{"@language":"en","@value":"Acceptance dataset"}]}]' | Set-Content -LiteralPath $jsonld
'<?xml version="1.0"?><rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dcterms="http://purl.org/dc/terms/"><rdf:Description rdf:about="https://example.org/acceptance"><dcterms:title xml:lang="en">Acceptance dataset</dcterms:title></rdf:Description></rdf:RDF>' | Set-Content -LiteralPath $rdf
foreach ($path in $ttl, $jsonld, $rdf) {
& $env:ACORN_BIN --offline --no-local-database check $path --standard dcmi --disable-website-checks --skip prose --no-fail *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Dublin Core check failed for $path" }
}
Write-Host "[PASS] Turtle, JSON-LD, and RDF/XML Dublin Core inputs were accepted" -ForegroundColor Green
CHECK-04 β Check text from standard input
"Text supplied through standard input. It has several words and another sentence." |
& $env:ACORN_BIN --offline --no-local-database check --all --skip prose,link,crosswalk,schema,quality
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Standard-input text check"
Expected: the report identifies the temporary source as stdin.txt.
CHECK-05 β Check a PDF as a document
& $env:ACORN_BIN --offline --no-local-database check "$env:ACORN_QA_INPUT\gather\gather.pdf" --standard document --disable-website-checks --skip prose --no-fail
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "PDF document check"
Expected: ACORN extracts text and reports document checks. Font-encoding warnings from PDF extraction are acceptable; a crash is not.
CHECK-06 β Verify terse, skip, and alternate-readability options
foreach ($metric in 'fkgl','ari','cli') {
& $env:ACORN_BIN --offline --no-local-database check "$env:ACORN_QA_INPUT\valid-rad\index.json" --disable-website-checks --skip prose,link --readability-test $metric --terse *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Readability metric failed: $metric" }
}
Write-Host "[PASS] Terse output, skipped checks, and three readability metrics work" -ForegroundColor Green
CHECK-07 β Reject a remote source in offline mode
& $env:ACORN_BIN --offline --no-local-database check "https://example.org/project.json" --skip prose
$code = $LASTEXITCODE
Confirm-NonzeroExitCode -Actual $code -Test "Offline mode rejects remote check input"
Expected: the error says the remote source cannot be used offline.
CHECK-08 β Check clipboard text and restore the clipboard
$previousClipboard = Get-Clipboard -Raw -ErrorAction SilentlyContinue
try {
Set-Clipboard -Value 'doi:10.1038/nature12373 is mentioned in this clipboard test.'
& $env:ACORN_BIN --offline --no-local-database check --paste --format text --standard text --skip prose,link,crosswalk,schema,quality
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Clipboard text check"
} finally {
Set-Clipboard -Value $previousClipboard
}
3. Format and link RAD
FORMAT-01 β Preview formatting without changing the file
$file = "$env:ACORN_QA_INPUT\format-rad\index.json"
$before = (Get-FileHash -LiteralPath $file -Algorithm SHA256).Hash
& $env:ACORN_BIN --no-local-database format $file --dry-run --no-color
$code = $LASTEXITCODE
$after = (Get-FileHash -LiteralPath $file -Algorithm SHA256).Hash
Confirm-ExitCode -Actual $code -Expected 0 -Test "Format dry run"
if ($before -ne $after) { throw "[FAIL] Dry run changed the file" }
Write-Host "[PASS] Dry run left the source unchanged" -ForegroundColor Green
Expected: a diff shows vocabulary normalization such as rs to rust, but the checksum stays unchanged.
FORMAT-02 β Apply formatting
$file = "$env:ACORN_QA_INPUT\format-rad\index.json"
& $env:ACORN_BIN --no-local-database format $file
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Format writes normalized RAD"
Confirm-Text -Path $file -Pattern '"rust"' -Test "Technology was normalized"
Confirm-Text -Path $file -Pattern 'Department of Energy' -Test "Sponsor was normalized"
FORMAT-03 β Verify offline policy is explicit
& $env:ACORN_BIN --offline --no-local-database format "$env:ACORN_QA_INPUT\valid-rad\index.json" --dry-run
$code = $LASTEXITCODE
Confirm-NonzeroExitCode -Actual $code -Test "Unsupported offline format mode is rejected clearly"
Expected in the current CLI: a clear message says offline mode is not implemented for format.
FORMAT-04 β Format and compare Dublin Core across syntaxes
$ttl = "$env:ACORN_QA_OUTPUT\dublin-core.ttl"
$jsonld = "$env:ACORN_QA_OUTPUT\dublin-core.jsonld"
& $env:ACORN_BIN --no-local-database format $ttl --standard dcmi --dry-run --no-color *> $null
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Dublin Core format preview"
& $env:ACORN_BIN --no-local-database diff $ttl $jsonld --standard dcmi
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Dublin Core cross-syntax semantic diff"
Expected: formatting succeeds without modifying the Turtle file, and the semantic diff is empty because both files describe the same graph.
LINK-01 β Preview linked-data output without changing files
$file = "$env:ACORN_QA_INPUT\link-rad\index.json"
$before = (Get-FileHash -LiteralPath $file -Algorithm SHA256).Hash
& $env:ACORN_BIN --no-local-database link $file --dry-run
$code = $LASTEXITCODE
$after = (Get-FileHash -LiteralPath $file -Algorithm SHA256).Hash
Confirm-ExitCode -Actual $code -Expected 0 -Test "Link dry run"
if ($before -ne $after) { throw "[FAIL] Link dry run changed the source" }
Write-Host "[PASS] Link preview left the source unchanged" -ForegroundColor Green
LINK-02 β Create JSON-LD
$file = "$env:ACORN_QA_INPUT\link-rad\index.json"
& $env:ACORN_BIN --no-local-database link $file
$code = $LASTEXITCODE
$jsonld = "$env:ACORN_QA_INPUT\link-rad\index.jsonld"
Confirm-ExitCode -Actual $code -Expected 0 -Test "JSON-LD creation"
Confirm-Path -Path $jsonld -Test "JSON-LD file exists"
Confirm-Text -Path $jsonld -Pattern 'schema.org' -Test "JSON-LD contains linked-data context"
4. Gather identifiers
GATHER-01 β Find and normalize identifiers in a text file
$report = "$env:ACORN_QA_OUTPUT\identifiers.raw.txt"
& $env:ACORN_BIN --offline --no-local-database gather "$env:ACORN_QA_INPUT\identifiers.txt" --raw | Tee-Object -FilePath $report
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Identifier discovery"
Confirm-Text -Path $report -Pattern '10.1038/nature12373' -Test "DOI was found"
Confirm-Text -Path $report -Pattern '0000-0002-1825-0097' -Test "ORCID was found"
Confirm-Text -Path $report -Pattern '2101.00001' -Test "arXiv identifier was found"
GATHER-02 β Produce JSON, YAML, and Markdown reports
foreach ($format in 'json','yaml','markdown') {
$destination = "$env:ACORN_QA_OUTPUT\gather.$format"
& $env:ACORN_BIN --offline --no-local-database gather "$env:ACORN_QA_INPUT\identifiers.txt" --format $format --output $destination
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Gather $format output failed" }
if (-not (Test-Path -LiteralPath $destination)) { throw "[FAIL] Gather did not create $destination" }
}
Write-Host "[PASS] Gather wrote JSON, YAML, and Markdown reports" -ForegroundColor Green
GATHER-03 β Extract identifiers from a PDF
& $env:ACORN_BIN --offline --no-local-database gather "$env:ACORN_QA_INPUT\gather\gather.pdf" --raw > "$env:ACORN_QA_OUTPUT\pdf-identifiers.txt"
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "PDF identifier gathering"
Confirm-Text -Path "$env:ACORN_QA_OUTPUT\pdf-identifiers.txt" -Pattern '10.1145/73560.73576' -Test "A DOI was extracted from the PDF"
GATHER-04 β Respect recursive depth
$tree = "$env:ACORN_QA_INPUT\depth-test"
New-Item -ItemType Directory -Force -Path "$tree\child\grandchild" | Out-Null
'doi:10.1000/direct' | Set-Content "$tree\direct.txt"
'doi:10.1000/child' | Set-Content "$tree\child\child.txt"
'doi:10.1000/grandchild' | Set-Content "$tree\child\grandchild\grandchild.txt"
& $env:ACORN_BIN --offline --no-local-database gather $tree --max-depth 1 --raw > "$env:ACORN_QA_OUTPUT\depth-one.txt"
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Gather depth limit"
Confirm-Text -Path "$env:ACORN_QA_OUTPUT\depth-one.txt" -Pattern '10.1000/direct' -Test "Direct child was included"
if (Select-String -LiteralPath "$env:ACORN_QA_OUTPUT\depth-one.txt" -Pattern 'grandchild' -Quiet) { throw "[FAIL] Grandchild exceeded max depth" }
Write-Host "[PASS] Deeper descendants were excluded" -ForegroundColor Green
GATHER-05 β Persist discoveries in an isolated SQLite database
& $env:ACORN_BIN --offline --database-backend sqlite --database-path "$env:ACORN_QA_DATABASE" gather --text "doi:10.1038/nature12373" --format json > "$env:ACORN_QA_OUTPUT\database-gather.json"
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "SQLite-backed gather"
Confirm-Path -Path $env:ACORN_QA_DATABASE -Test "SQLite database was created"
5. Export and publish
EXPORT-01 β Export RAD to YAML in a chosen output directory
$destination = "$env:ACORN_QA_OUTPUT\yaml"
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\valid-rad\index.json" --format yaml --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "YAML export command"
Confirm-Path -Path "$destination\index.yaml" -Test "YAML was written to the requested directory"
Expected: output/yaml/index.yaml exists. If ACORN writes beside the source instead, record a failure because --output was not honored.
EXPORT-02 β Export RAD to Markdown
$destination = "$env:ACORN_QA_OUTPUT\markdown"
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\valid-rad\index.json" --format markdown --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Markdown export command"
Confirm-Path -Path "$destination\index.md" -Test "Markdown was written to the requested directory"
Confirm-Text -Path "$destination\index.md" -Pattern 'Valid Project' -Test "Markdown contains the project title"
EXPORT-03 β Create and validate CITATION.cff
$source = "$env:ACORN_QA_INPUT\valid-rad\index.json"
& $env:ACORN_BIN --offline --no-local-database export $source --format cff
$exportCode = $LASTEXITCODE
$cff = "$env:ACORN_QA_INPUT\valid-rad\CITATION.cff"
Confirm-ExitCode -Actual $exportCode -Expected 0 -Test "CFF export"
Confirm-Path -Path $cff -Test "CITATION.cff exists beside the source"
& $env:ACORN_BIN --offline --no-local-database check $cff --disable-website-checks --skip prose --no-fail
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Generated CFF can be checked"
EXPORT-04 β Export JSON Schemas for supported standards
$standards = 'acre','rads','cff','datacite','dcat','dcmi','invenio','huwise','raid'
foreach ($standard in $standards) {
$schema = "$env:ACORN_QA_OUTPUT\$standard.schema.json"
& $env:ACORN_BIN --offline --no-local-database export --schema --standard $standard --format json --raw > $schema
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Schema export failed for $standard" }
if ((Get-Item -LiteralPath $schema).Length -lt 100) { throw "[FAIL] Schema was unexpectedly small for $standard" }
Get-Content -LiteralPath $schema -Raw | ConvertFrom-Json | Out-Null
}
Write-Host "[PASS] Nine supported standards produced valid JSON Schemas" -ForegroundColor Green
Expected: ACRE, RAD, CFF, DataCite, DCAT, Dublin Core, InvenioRDM, HuWise, and RAiD schemas are non-empty JSON.
EXPORT-05 β Crosswalk DataCite to DCAT
$destination = "$env:ACORN_QA_OUTPUT\crosswalk"
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\schema\datacite.json" --format json --from datacite --to dcat --output $destination
$code = $LASTEXITCODE
$result = "$destination\datacite.json"
Confirm-ExitCode -Actual $code -Expected 0 -Test "DataCite-to-DCAT crosswalk"
Confirm-Path -Path $result -Test "Crosswalk output exists"
Confirm-Text -Path $result -Pattern 'Example Dataset Title' -Test "Crosswalk retained the title"
EXPORT-05A β Crosswalk DataCite to Dublin Core Turtle
$destination = "$env:ACORN_QA_OUTPUT\crosswalk-dublin-core"
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\schema\datacite.json" --format turtle --from datacite --to dcmi --output $destination
$code = $LASTEXITCODE
$result = "$destination\datacite.ttl"
Confirm-ExitCode -Actual $code -Expected 0 -Test "DataCite-to-Dublin-Core crosswalk"
Confirm-Path -Path $result -Test "Dublin Core Turtle exists"
Confirm-Text -Path $result -Pattern 'purl.org/dc/terms' -Test "Turtle uses DCMI terms"
& $env:ACORN_BIN --offline --no-local-database check $result --standard dcmi --disable-website-checks --skip prose --no-fail *> $null
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Crosswalked Dublin Core can be checked"
EXPORT-05B β Exercise every supported crosswalk pairing
$sources = @{
datacite = 'datacite.json'
dcat = 'dcat.json'
invenio = 'invenio-single.json'
huwise = 'huwise.json'
}
$failures = @()
foreach ($from in $sources.Keys) {
foreach ($to in $sources.Keys | Where-Object { $_ -ne $from }) {
$destination = "$env:ACORN_QA_OUTPUT\crosswalk-$from-to-$to"
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\schema\$($sources[$from])" --format json --from $from --to $to --output $destination *> $null
if ($LASTEXITCODE -ne 0) { $failures += "$from -> $to" }
}
}
if ($failures) { throw "[FAIL] Crosswalk pairings failed: $($failures -join ', ')" }
Write-Host "[PASS] All 12 crosswalk pairings completed" -ForegroundColor Green
EXPORT-05C β Import Dublin Core into RAD with explicit local context
$source = "$env:ACORN_QA_OUTPUT\crosswalk-dublin-core\datacite.ttl"
$contact = "$env:ACORN_QA_OUTPUT\dublin-core-contact.json"
$destination = "$env:ACORN_QA_OUTPUT\dublin-core-rad"
(Get-Content "$env:ACORN_QA_INPUT\valid-rad\index.json" -Raw | ConvertFrom-Json).contact |
ConvertTo-Json -Depth 20 | Set-Content -LiteralPath $contact
New-Item -ItemType Directory -Force -Path $destination | Out-Null
& $env:ACORN_BIN --offline --no-local-database export $source --format json --from dcmi --to rads --crosswalk-contact $contact --local-identifier dcmi-acceptance --output $destination
$code = $LASTEXITCODE
$result = "$destination\datacite.json"
Confirm-ExitCode -Actual $code -Expected 0 -Test "Dublin Core-to-RAD import"
Confirm-Path -Path $result -Test "Imported RAD exists"
Confirm-Text -Path $result -Pattern 'dcmi-acceptance' -Test "Caller-supplied local identifier was retained"
Expected: the import succeeds only because the tester supplied both a local contact and identifier. Running the same command without either required option must fail before writing output.
EXPORT-06 β Create a PDF fact sheet
Skip if setup printed Browser: not detected.
if (-not $env:ACORN_QA_CHROME) {
Write-Host "[SKIP] PDF export: Chrome or Edge was not detected" -ForegroundColor Yellow
} else {
$destination = "$env:ACORN_QA_OUTPUT\pdf"
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\highlight\index.json" --format pdf --chrome-path $env:ACORN_QA_CHROME --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "PDF export"
$pdf = Get-ChildItem -LiteralPath $destination -Filter *.pdf | Select-Object -First 1
if (-not $pdf -or $pdf.Length -lt 10000) { throw "[FAIL] PDF is missing or unexpectedly small" }
Write-Host "[PASS] PDF exists and is non-empty: $($pdf.FullName)" -ForegroundColor Green
}
Manual visual check: open the PDF and confirm text is readable, images are not stretched, no content is cut off, and any ASPECT chart has legible labels.
EXPORT-07 β Create a PowerPoint from a reference template
$destination = "$env:ACORN_QA_OUTPUT\powerpoint"
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\highlight\index.json" --format powerpoint --reference "$env:ACORN_QA_INPUT\highlight\reference.pptx" --show-aspect-labels --show-aspect-scores --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "PowerPoint export"
$pptx = Get-ChildItem -LiteralPath $destination -Filter *.pptx | Select-Object -First 1
if (-not $pptx -or $pptx.Length -lt 10000) { throw "[FAIL] PowerPoint is missing or unexpectedly small" }
Write-Host "[PASS] PowerPoint exists and is non-empty: $($pptx.FullName)" -ForegroundColor Green
Manual visual check: open the presentation and confirm placeholders were replaced, the image appears, speaker notes are sensible, and PowerPoint does not request a repair.
EXPORT-08 β Create verified BagIt archives
$archives = @(
@('zip','deposit.zip'),
@('7z','deposit.7z'),
@('tar','deposit.tar'),
@('tar.gz','deposit.tar.gz')
)
$failures = @()
foreach ($archiveCase in $archives) {
$archive = "$env:ACORN_QA_OUTPUT\$($archiveCase[1])"
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\bag-source" --format bag --archive-format $archiveCase[0] --output $archive *> $null
if ($LASTEXITCODE -ne 0 -or -not (Test-Path -LiteralPath $archive)) { $failures += $archiveCase[0] }
}
if ($failures) { throw "[FAIL] BagIt archive formats failed: $($failures -join ', ')" }
Write-Host "[PASS] ZIP, 7z, TAR, and TAR.GZ BagIt exports passed verification" -ForegroundColor Green
Expected: the archive is created and ACORNβs built-in post-export verification passes. A ZIP file left behind after an error is still a failure.
EXPORT-09 β Preview export without writing
$destination = "$env:ACORN_QA_OUTPUT\dry-export"
& $env:ACORN_BIN --offline --no-local-database export "$env:ACORN_QA_INPUT\valid-rad\index.json" --format pdf --output $destination --dry-run
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Export dry run"
if (Test-Path -LiteralPath $destination) { throw "[FAIL] Export dry run created output" }
Write-Host "[PASS] Export dry run wrote no artifact" -ForegroundColor Green
EXPORT-10 β Combine multiple RAD records into one JSON artifact
$source = "$env:ACORN_QA_INPUT\combine"
$destination = "$env:ACORN_QA_OUTPUT\combined"
New-Item -ItemType Directory -Force -Path $source, $destination | Out-Null
Copy-Item "$env:ACORN_QA_INPUT\valid-rad\index.json" "$source\one.json"
Copy-Item "$env:ACORN_QA_INPUT\highlight\index.json" "$source\two.json"
& $env:ACORN_BIN --offline --no-local-database export $source --format json --combine --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Combined JSON export"
$combined = Get-ChildItem -LiteralPath $destination -Filter *.json | Select-Object -First 1
if (-not $combined) { throw "[FAIL] Combined export did not create one JSON artifact" }
$records = Get-Content -LiteralPath $combined.FullName -Raw | ConvertFrom-Json
if (@($records).Count -lt 2) { throw "[FAIL] Combined artifact has fewer than two records" }
Write-Host "[PASS] Combined artifact contains multiple records" -ForegroundColor Green
6. Import research content and API specifications
IMPORT-01 β Import a local research-content directory
$destination = "$env:ACORN_QA_OUTPUT\imported"
& $env:ACORN_BIN --offline --no-local-database import "$env:ACORN_QA_INPUT\valid-rad" --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Local content import"
Confirm-Path -Path "$destination\index.json" -Test "Imported RAD exists"
IMPORT-02 β Filter and flatten an import
$destination = "$env:ACORN_QA_OUTPUT\flattened"
& $env:ACORN_BIN --offline --no-local-database import "$env:ACORN_QA_INPUT\valid-rad" --filter 'index\.json$' --flatten --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Filtered, flattened import"
Confirm-Path -Path "$destination\index.json" -Test "Selected file was flattened"
if (Test-Path -LiteralPath "$destination\00.png") { throw "[FAIL] Filter allowed an unwanted image" }
Write-Host "[PASS] Filter excluded unrelated files" -ForegroundColor Green
IMPORT-03 β Refuse to overwrite unless --clobber is present
$destination = "$env:ACORN_QA_OUTPUT\clobber-test"
& $env:ACORN_BIN --offline --no-local-database import "$env:ACORN_QA_INPUT\valid-rad" --output $destination *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Initial import failed" }
& $env:ACORN_BIN --offline --no-local-database import "$env:ACORN_QA_INPUT\valid-rad" --output $destination *> $null
$withoutClobber = $LASTEXITCODE
Confirm-NonzeroExitCode -Actual $withoutClobber -Test "Import protects existing content"
& $env:ACORN_BIN --offline --no-local-database import "$env:ACORN_QA_INPUT\valid-rad" --output $destination --clobber *> $null
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Import clobber replaces selected content"
IMPORT-04 β Generate an endpoint resource from OpenAPI
$preview = "$env:ACORN_QA_OUTPUT\endpoint-preview.json"
& $env:ACORN_BIN --offline --no-local-database import spec "$env:ACORN_QA_INPUT\openapi.yaml" --name qa::api --domain api.example.org --dry-run > $preview
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "OpenAPI dry-run import"
Confirm-Text -Path $preview -Pattern 'list-projects' -Test "Endpoint operation was imported"
$written = "$env:ACORN_QA_OUTPUT\endpoint.json"
& $env:ACORN_BIN --offline --no-local-database import spec "$env:ACORN_QA_INPUT\openapi.yaml" --name qa::api --domain api.example.org --output $written
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "OpenAPI file import"
Confirm-Path -Path $written -Test "Endpoint JSON was written"
IMPORT-05 β Re-import the BagIt archive
Run this only after EXPORT-08 passes.
$archive = "$env:ACORN_QA_OUTPUT\deposit.zip"
if (-not (Test-Path -LiteralPath $archive)) {
Write-Host "[SKIP] BagIt import: EXPORT-08 did not produce an archive" -ForegroundColor Yellow
} else {
$destination = "$env:ACORN_QA_OUTPUT\bag-import"
& $env:ACORN_BIN --offline --no-local-database import $archive --output $destination
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "BagIt archive import and checksum verification"
Confirm-Path -Path "$destination\index.json" -Test "BagIt payload was imported"
}
IMPORT-06 β Read JSON, JSONC, and YAML configuration files
$configs = @(
"$env:ACORN_REPO\tests\fixtures\config\with_local_buckets.json",
"$env:ACORN_REPO\tests\fixtures\config\with_buckets.jsonc",
"$env:ACORN_QA_CONFIG\local.yaml"
)
for ($index = 0; $index -lt $configs.Count; $index++) {
$destination = "$env:ACORN_QA_OUTPUT\configured-$index"
& $env:ACORN_BIN --offline --no-local-database import --config $configs[$index] --output $destination *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Configuration import failed: $($configs[$index])" }
if (-not (Test-Path -LiteralPath $destination)) { throw "[FAIL] Configuration produced no output" }
}
Write-Host "[PASS] JSON, JSONC, and YAML configurations were accepted" -ForegroundColor Green
7. Software identifiers
SWHID-01 β Hash the same file deterministically
$file = "$env:ACORN_QA_INPUT\identifiers.txt"
$first = (& $env:ACORN_BIN --no-local-database swhid $file --raw | Out-String).Trim()
$firstCode = $LASTEXITCODE
$second = (& $env:ACORN_BIN --no-local-database swhid $file --raw | Out-String).Trim()
$secondCode = $LASTEXITCODE
if ($firstCode -ne 0 -or $secondCode -ne 0) { throw "[FAIL] File SWHID calculation failed" }
if ($first -ne $second -or $first -notmatch '^swh:1:cnt:[0-9a-f]{40}$') { throw "[FAIL] File SWHID was not deterministic or valid" }
Write-Host "[PASS] Deterministic file SWHID: $first" -ForegroundColor Green
SWHID-02 β Verify a calculated identifier
$file = "$env:ACORN_QA_INPUT\identifiers.txt"
$id = (& $env:ACORN_BIN --no-local-database swhid $file --raw | Out-String).Trim()
& $env:ACORN_BIN --no-local-database swhid $file --verify $id
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "SWHID verification"
SWHID-03 β Hash directory and Git objects
& $env:ACORN_BIN --no-local-database swhid "$env:ACORN_QA_INPUT\valid-rad" --kind directory --raw
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Directory SWHID failed" }
$repository = "$env:ACORN_QA_ROOT\swhid-git"
New-Item -ItemType Directory -Force -Path $repository | Out-Null
Copy-Item "$env:ACORN_QA_INPUT\valid-rad\index.json" "$repository\index.json"
Push-Location $repository
try {
git init --quiet
git config user.email 'acorn-qa@example.org'
git config user.name 'ACORN QA'
git add index.json
git commit --quiet -m 'Add RAD fixture'
git tag -a v1.0.0 -m 'Acceptance test release'
$gitCases = @(
@('git-blob','HEAD:index.json'),
@('git-tree','HEAD'),
@('revision','HEAD'),
@('release','v1.0.0')
)
foreach ($case in $gitCases) {
& $env:ACORN_BIN --no-local-database swhid . --kind $case[0] --reference $case[1] --raw
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Git SWHID failed for $($case[0])" }
}
& $env:ACORN_BIN --no-local-database swhid . --kind snapshot --raw
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Git snapshot SWHID failed" }
} finally {
Pop-Location
}
Write-Host "[PASS] Directory, blob, tree, revision, release, and snapshot SWHIDs were calculated" -ForegroundColor Green
8. Diagnostics, model configuration, and agent support
DOCTOR-01 β Run all diagnostics
& $env:ACORN_BIN --no-local-database doctor
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Doctor diagnostics"
Expected: system, memory, network, GPU, and software findings are shown. Missing optional software is a diagnostic result, not necessarily a test failure.
DOCTOR-02 β Produce a machine-readable report
$report = "$env:ACORN_QA_OUTPUT\doctor.json"
& $env:ACORN_BIN --no-local-database doctor --report > $report
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "Doctor JSON report"
Get-Content -LiteralPath $report -Raw | ConvertFrom-Json | Out-Null
Write-Host "[PASS] Doctor report is valid JSON" -ForegroundColor Green
SYNC-01 β Preview VS Code model configuration
$target = "$env:ACORN_QA_OUTPUT\chatLanguageModels.json"
& $env:ACORN_BIN --no-local-database sync --model-file "$env:ACORN_QA_INPUT\models.txt" --assume-models --vscode --force --vscode-config $target --dry-run --no-color
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "VS Code sync preview"
if (Test-Path -LiteralPath $target) { throw "[FAIL] Sync dry run wrote the target file" }
Write-Host "[PASS] Sync preview made no changes" -ForegroundColor Green
SYNC-02 β Write isolated app configurations
$models = "$env:ACORN_QA_INPUT\models.txt"
$targets = @(
@('--vscode','--vscode-config',"$env:ACORN_QA_OUTPUT\vscode.json"),
@('--opencode','--opencode-config',"$env:ACORN_QA_OUTPUT\opencode.jsonc"),
@('--llama-swap','--llama-swap-config',"$env:ACORN_QA_OUTPUT\llama-swap.yaml"),
@('--goose','--goose-config',"$env:ACORN_QA_OUTPUT\goose.yaml")
)
foreach ($target in $targets) {
& $env:ACORN_BIN --no-local-database sync --model-file $models --assume-models --force @target *> $null
if ($LASTEXITCODE -ne 0) { throw "[FAIL] Sync failed for $($target[0])" }
if (-not (Test-Path -LiteralPath $target[2])) { throw "[FAIL] Sync did not write $($target[2])" }
}
Write-Host "[PASS] Four isolated application configurations were written" -ForegroundColor Green
SKILL-01 β Extract ACORNβs agent instructions
This writes one cached skill file and may update the clipboard.
$skillOutput = & $env:ACORN_BIN --no-local-database skill 2>&1 | Out-String
$code = $LASTEXITCODE
$skillOutput
Confirm-ExitCode -Actual $code -Expected 0 -Test "Agent skill extraction"
if ($skillOutput -notmatch 'SKILL\.md') { throw "[FAIL] Skill output did not identify SKILL.md" }
9. Interactive and watch behavior
These are manual because they intentionally keep running until the tester stops them.
WATCH-01 β Re-run a check when a file changes
Run the first block, wait for the initial report, then open another PowerShell window and run the second block. Return to the first window and press q or Ctrl+C after the second report.
First window:
& $env:ACORN_BIN --offline --no-local-database --watch --watch-mode poll --poll-interval 1s check "$env:ACORN_QA_INPUT\valid-rad\index.json" --disable-website-checks --skip prose --no-fail
Second window:
(Get-Item "$env:ACORN_QA_INPUT\valid-rad\index.json").LastWriteTime = Get-Date
Expected: exactly one initial run and at least one new run after the timestamp changes; stopping returns control to the prompt.
TUI-01 β Open and exit the terminal interface
& $env:ACORN_BIN --offline --database-path "$env:ACORN_QA_DATABASE" tui
Expected: a readable interactive screen opens, navigation keys respond, and the displayed quit key exits cleanly. Resize the terminal once and confirm the interface redraws without corruption.
GIT-01 β Select files changed by a Git commit
This creates a tiny disposable Git repository; it does not touch the ACORN repositoryβs history.
$gitTest = "$env:ACORN_QA_ROOT\git-selector"
New-Item -ItemType Directory -Force -Path $gitTest | Out-Null
Copy-Item "$env:ACORN_QA_INPUT\valid-rad\index.json" "$gitTest\index.json"
Push-Location $gitTest
try {
git init --quiet
git config user.email 'acorn-qa@example.org'
git config user.name 'ACORN QA'
git add index.json
git commit --quiet -m 'Add RAD fixture'
(Get-Content index.json -Raw).Replace('Valid Project','Valid Project Updated') | Set-Content index.json
git add index.json
git commit --quiet -m 'Update RAD fixture'
& $env:ACORN_BIN --offline --no-local-database check --commit HEAD --disable-website-checks --skip prose --no-fail
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Git commit selector"
} finally {
Pop-Location
}
Expected: the report includes index.json. A release that changes source selection should repeat this pattern for format --dry-run, link --dry-run, and export --dry-run.
10. Optional connected tests
Run these only on a normal internet connection. They can be slower or depend on third-party uptime.
NET-01 β Check live links and synchronize the prose analyzer
& $env:ACORN_BIN --no-local-database check "$env:ACORN_QA_INPUT\valid-rad\index.json" --all --no-fail
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Connected prose and link checks"
Expected: ACORN runs schema, prose, readability, quality, and website checks. A deliberately invalid fixture URL may be reported as a content problem.
NET-02 β Resolve a DOI citation
& $env:ACORN_BIN --no-local-database gather --text "doi:10.1038/nature12373" --resolve --raw --citation-format apa > "$env:ACORN_QA_OUTPUT\citation.txt"
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "DOI resolution"
Confirm-Text -Path "$env:ACORN_QA_OUTPUT\citation.txt" -Pattern '2013' -Test "Resolved citation contains publication details"
NET-03 β Search DOE CODE
& $env:ACORN_BIN --no-local-database gather --osti projects "ACORN" --lab ORNL --limit 5 --format json > "$env:ACORN_QA_OUTPUT\osti.json"
$code = $LASTEXITCODE
Confirm-ExitCode -Actual $code -Expected 0 -Test "DOE CODE project search"
Get-Content -LiteralPath "$env:ACORN_QA_OUTPUT\osti.json" -Raw | ConvertFrom-Json | Out-Null
Write-Host "[PASS] DOE CODE returned valid JSON" -ForegroundColor Green
NET-04 β Preview model metadata without downloading weights
& $env:ACORN_BIN --no-local-database import model Qwen/Qwen2.5-0.5B-Instruct-GGUF --no-fallback --dry-run
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Hugging Face model metadata preview"
Expected: GGUF filenames, quantizations, revisions, and sizes are reported; no model weight is downloaded.
NET-05 β Preview model download selection
& $env:ACORN_BIN --no-local-database download model Qwen/Qwen2.5-0.5B-Instruct-GGUF --quantization Q4_K_M --local-dir "$env:ACORN_QA_OUTPUT\models" --dry-run --raw
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Model download dry run"
Expected: structured metadata describes the selected files; the models directory remains empty. Remove --dry-run only with owner approval because model files can be large.
NET-06 β Download a filtered remote bucket
& $env:ACORN_BIN --no-local-database download https://github.com/jhwohlgemuth/bucket --filter 'index\.json$' --output "$env:ACORN_QA_OUTPUT\remote-bucket"
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Filtered remote bucket download"
Expected: matching research index files are downloaded without unrelated large content. A third-party repository move or outage should be recorded as SKIP β external dependency, not an ACORN pass.
NET-07 β Preview provider enrichment
& $env:ACORN_BIN --no-local-database format "$env:ACORN_QA_INPUT\valid-rad\index.json" --enrich --provider openalex,citeas --dry-run --no-color
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Metadata enrichment preview"
Expected: existing populated values remain intact and any provider additions appear only in the preview.
NET-08 β Canonicalize and strictly ingest a remote bucket
& $env:ACORN_BIN --database-path "$env:ACORN_QA_DATABASE" download https://github.com/jhwohlgemuth/bucket --filter 'index\.json$' --canonicalize --strict --output "$env:ACORN_QA_OUTPUT\canonical-bucket"
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Canonical, strict bucket ingestion"
Expected: deprecated aliases are rewritten before strict database ingestion. If the external test repository contains no deprecated aliases, the test still verifies that the combined mode completes.
11. Owner-supervised infrastructure tests
Do not give these steps to an unsupervised tester. They need real GitLab credentials and/or a container runtime and may change external state.
INFRA-01 β Confirm create/serve argument validation without changing anything
& $env:ACORN_BIN create runner --help *> $null
$runnerHelp = $LASTEXITCODE
& $env:ACORN_BIN create bot --help *> $null
$createBotHelp = $LASTEXITCODE
& $env:ACORN_BIN serve bot --help *> $null
$serveBotHelp = $LASTEXITCODE
if (@($runnerHelp, $createBotHelp, $serveBotHelp) | Where-Object { $_ -ne 0 }) { throw "[FAIL] Infrastructure help failed" }
Write-Host "[PASS] Runner and bot interfaces are available without side effects" -ForegroundColor Green
INFRA-02 β Create a GitLab runner (owner performs substitutions)
& $env:ACORN_BIN create runner --group '<GITLAB_GROUP_ID>' --name 'acorn-qa-runner' --runtime docker --repo '<GITLAB_DOMAIN>'
Expected: a runner is registered and its container starts. The owner must remove the runner and container after the test.
INFRA-03 β Create and run a detached GitLab bot (owner performs substitutions)
& $env:ACORN_BIN create bot '<GITLAB_PROJECT_ID>' --name 'acorn-qa-bot' --runtime docker --detach --domain '<GITLAB_DOMAIN>'
Expected: the container starts, authenticates, and polls without exposing secrets. The owner must remove the test bot/container afterward.
INFRA-04 β Run the in-process bot service (owner performs substitutions)
& $env:ACORN_BIN serve bot '<GITLAB_PROJECT_ID>' --event-source poll --poll-interval 10
Expected: the service starts, handles a test note event once, and stops cleanly with Ctrl+C.
Webhook and hybrid delivery should be tested only with an approved HTTPS test URL, project, webhook token, and signing token. Never expose a development machine directly to the public internet for this test.
12. Explicitly incomplete interfaces
These tests document intentional gaps. They pass when the CLI rejects the request clearly and non-destructively.
GAP-01 β create mcp reports that it is not implemented
& $env:ACORN_BIN --no-local-database create mcp
Confirm-NonzeroExitCode -Actual $LASTEXITCODE -Test "create mcp reports an implementation gap"
MCP-01 β serve mcp starts the stdio protocol server
@'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"acceptance","version":"1"}}}'@ | & $env:ACORN_BIN --no-local-database serve mcp
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "serve mcp protocol initialization"
DOCTOR-03 β Doctor safe autofix remains bounded
& $env:ACORN_BIN --no-local-database doctor --fix
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Doctor safe autofix"
13. Global behavior and safety
GLOBAL-01 β Verify quiet and verbose modes
$quiet = & $env:ACORN_BIN --offline --no-local-database gather --text "doi:10.1038/nature12373" --quiet 2>&1 | Out-String
$quietCode = $LASTEXITCODE
$verbose = & $env:ACORN_BIN --offline --no-local-database gather --text "doi:10.1038/nature12373" -vv 2>&1 | Out-String
$verboseCode = $LASTEXITCODE
if ($quietCode -ne 0 -or $verboseCode -ne 0) { throw "[FAIL] Quiet or verbose command failed" }
if ($quiet.Length -ge $verbose.Length) { throw "[FAIL] Quiet output was not smaller than verbose output" }
Write-Host "[PASS] Quiet and verbose levels produce appropriately different output" -ForegroundColor Green
GLOBAL-02 β Verify thread-count parsing
& $env:ACORN_BIN --offline --no-local-database --threads 1 gather --text "doi:10.1038/nature12373" --raw *> $null
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Single-thread mode"
GLOBAL-03 β Reject conflicting database flags
& $env:ACORN_BIN --database-path "$env:ACORN_QA_DATABASE" --no-local-database doctor
Confirm-NonzeroExitCode -Actual $LASTEXITCODE -Test "Conflicting database options are rejected"
GLOBAL-04 β Request the optional DuckDB backend in isolation
$duckdb = "$env:ACORN_QA_ROOT\database\acceptance.duckdb"
& $env:ACORN_BIN --offline --database-backend duckdb --database-path $duckdb gather --text "doi:10.1038/nature12373" --format json > "$env:ACORN_QA_OUTPUT\duckdb-gather.json" 2> "$env:ACORN_QA_OUTPUT\duckdb-error.txt"
$code = $LASTEXITCODE
if ($code -eq 0) {
Confirm-Path -Path $duckdb -Test "DuckDB database was created"
} elseif (Select-String -LiteralPath "$env:ACORN_QA_OUTPUT\duckdb-error.txt" -Pattern "not available in this build" -Quiet) {
Write-Host "[PASS] This build clearly reports that optional DuckDB support is unavailable" -ForegroundColor Green
} else {
throw "[FAIL] DuckDB neither worked nor reported the expected feature-gate message"
}
GLOBAL-05 β Reset only the disposable SQLite database
Run after GATHER-05 and before cleanup.
$resolvedDatabase = [System.IO.Path]::GetFullPath($env:ACORN_QA_DATABASE)
$allowedDatabaseRoot = [System.IO.Path]::GetFullPath("$env:ACORN_QA_ROOT\database")
if (-not $resolvedDatabase.StartsWith($allowedDatabaseRoot, [System.StringComparison]::OrdinalIgnoreCase)) {
throw "[FAIL] Refusing to reset a database outside the QA workspace"
}
& $env:ACORN_BIN --offline --database-path $resolvedDatabase --reset-database gather --text "doi:10.1038/nature12373" --format json > "$env:ACORN_QA_OUTPUT\after-reset.json"
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test "Disposable database reset"
Do not test --clear-cache against a normal user profile: it intentionally removes ACORN cache data and downloaded Chromium artifacts. Test --reset-database only against $env:ACORN_QA_DATABASE and only when earlier database tests no longer need their data.
14. Intake demo release gate
These cases cover the current SharePoint workflow: browser download of the collected JSONL, local staging and review, confirmed application, immutable corrections, and a combined PDF. They do not require a premium connector, direct OAuth download, Vergil, Git credentials, or a live cloud write. Run them against the exact binary intended for release. Python 3.9 or later is required only for this acceptance runner; it uses the standard library.
The dated result in tests/acceptance/intake-demo-results.json identifies the tested source commit, executable and runner hashes, platform, local acceptance results, and remaining live check. It is evidence for that build; rerun the rehearsal against the final minor-release executable.
Run from the repository root after SETUP-02. The runner creates a fresh child workspace, refuses existing destinations, copies fictional project metadata, and generates synthetic receipts. Its intake config uses explicit local file URIs; its report config resolves relative project paths with options.folders.root: "file:.". It records version, executable hash, operating system, command output, and every exit status in evidence.json and numbered transcripts. A failed check exits nonzero and retains the evidence. It never edits the source fixtures. Use only the synthetic fixture for the runnerβs automatic --yes and --force approvals.
$intakeRoot = Join-Path $env:ACORN_QA_ROOT 'intake-demo'
python scripts/qa/verify-intake-demo.py --acorn $env:ACORN_BIN --workspace $intakeRoot --chrome $env:ACORN_QA_CHROME
Confirm-ExitCode -Actual $LASTEXITCODE -Expected 0 -Test 'Intake demo automated checks'
If Chrome was not detected, supply its installed executable path, or omit --chrome to use ACORNβs browser discovery. Offline mode will not download a browser. --skip-pdf runs INTAKE-01β08 but records INTAKE-09 as skipped; that is not a passed demo gate. On macOS/Linux, the same runner accepts --acorn target/debug/acorn. Use a new workspace for each rerun.
INTAKE-01 β Stage and inspect three project receipts
The runner stages three synthetic submissions from a local JSONL file and inspects the returned batch ID. Expect three accepted receipts, all three project routes and their proposed changes in the inspection, unchanged source bytes, and unchanged destination indexes. JSONL is an envelope per line, including submission UUID, submitter, project, submission time, and entry; it is not a raw list of LogbookEntry values.
INTAKE-02 β Require approval before applying
The runner first invokes apply with redirected input/output and no --yes. Expect a nonzero exit explaining the terminal requirement and no index changes. It then approves only its reviewed synthetic batch with --yes. Expect one new entry in each project and intact quotes, Unicode, supporting items, and embedded newline text. Human button behavior is checked separately in INTAKE-11.
INTAKE-03 β Replay without duplicate entries
Repeat apply with the original batch ID, then restage the exact original receipts. Expect three replayed receipts, zero newly accepted receipts, and byte-identical indexes after both apply attempts. Keep the downloaded source; applying is not an inbox drain.
INTAKE-04 β Edit an existing submission through a new receipt
Apply a correction in a later batch using the original receipt evidence retained in the workspace. Inspection must identify the original and immediate predecessor. Expect the same logical entry identifier, corrected text, and no additional entry. Restage the complete original-plus-correction history: all four receipts replay and the corrected indexes stay byte-identical. In another fresh workspace, importing the complete history in one batch must produce the same indexes. Original receipts remain unchanged in the downloaded JSONL.
INTAKE-05 β Reject invalid or ambiguous history without partial writes
The runner exercises missing ancestry, a changed owner who is otherwise an authorized member, routing to another registered project, changed report placement, competing successors, changed bytes under an existing UUID, and a malformed envelope. Each has a separate workspace. Expect a rejected batch, a nonzero apply exit, and no changes to any project, including valid rows in the same batch.
INTAKE-06 β Recheck reviewed inputs at apply time
After staging, independently change the source bytes, destination index bytes, or authorized membership. Expect each apply attempt to fail and preserve all indexes exactly as they were immediately before apply. Review and stage a new batch after resolving the cause; do not bypass the digest checks.
INTAKE-07 β Keep the standalone general destination an explicit gap
Stage a mixed batch of valid project highlights plus a report-wide receipt routed to general, with global: true, and no associated project/index. Expect the general row to be rejected, the whole batch to be non-applicable, and no index writes. Preserve every receipt for later review; never silently drop the general row or route it to a project with fabricated contact information.
Current demo boundary: project highlights and their corrections can be demonstrated now. Standalone report-wide intake/reporting remains PA-29 work. Passing this negative case documents the limitation; it does not establish general-update support. Project-backed entries with global: true are a different, already supported report feature. If the planned demonstration includes standalone general updates, this gap blocks that demonstration.
INTAKE-08 β Preview and graduate optional metadata safely
In a separate workspace, ingest a synthetic personnel entry. intake run --dry-run --force must preview one contributor without writing. Applying the reviewed promotion with --force --expected-digest must add that contributor, retain the source entry as archived, and create an exact-byte backup. Reusing the stale digest must fail without further changes. Graduation is optional and separate from producing the sponsor-highlights PDF.
INTAKE-09 β Produce and inspect the combined PDF
The runner uses the corrected three-project indexes, an explicit October 26β27 reporting window, the default report root/output, and --dry-run --combine. No commit selector, publication, callback, or remote inbox update is needed. Expect demo/.acorn/reports/report.pdf and unchanged indexes. The PDF must open successfully; its mere existence is insufficient.
Start-Process (Join-Path $intakeRoot 'demo/.acorn/reports/report.pdf')
Inspect every page. Require a readable title page, all three project sections, and the legend. The watershed section must show TEST ONLY corrected watershed highlight exactly once and must not show its original highlight. The other two original highlights must each appear once. October 3 baseline entries must be absent from this October 26 window. Check text wrapping, quotes/Unicode, bullets, page breaks, colors, and absence of clipped or overlapping content. Record the visual result with the PDF hash from evidence.json; the runner deliberately leaves visualReview as PENDING until a person reviews it.
The actual demo sequence, from a directory with matching local configs, is:
acorn --offline intake stage --batch --config intake.json --source collected.jsonl
acorn --offline intake inspect --config intake.json --batch-id <batch-ID>
acorn --offline intake apply --config intake.json --batch-id <batch-ID>
acorn --offline report --config report.json --from 2026-10-26 --through 2026-10-27 --dry-run --combine
Replace the dates and batch ID for the actual demo. Keep original and correction rows together, or retain previously applied ancestry in the same workspace. The terminalβs Yes/No confirmation is the normal operator path; the automated runner uses explicit approvals only for its fictional data.
INTAKE-10 β Smoke-test the live form-to-download handoff
This is a separate manual, authorized cloud check. Local synthetic results do not prove current tenant access or delivery. Before demonstrating the live form:
- Use a standard-license contributor to open the SharePoint form. Confirm Project highlight is the default. Briefly select Report-wide update (no associated project): project controls should be disabled. Switch back, load projects, and submit a clearly labeled TEST ONLY project highlight. Require visible success/receipt feedback and record its UUID. My submissions should occupy its own row below the banner without overlapping it.
- Confirm the contributor and operator receive Teams confirmations, with no routine submission email. Keep email reserved for the configured milestone workflow.
- Open My submissions; verify the contributor sees only their own history. Edit the test highlight; project and report placement must remain fixed. Save the correction, require visible success feedback, record its new UUID, and verify the Teams confirmation.
- Wait for the collected export to refresh, download
logbook.jsonl, and confirm it contains both immutable receipts with the correct root/predecessor links. A receipt download is not the collected export. Confirm other existing rows were retained; use the retained source snapshot to compare. - Stage only the exact selected project receipts and their complete ancestry into a separate local workspace, inspect, approve, and generate the PDF. Verify the corrected highlight appears once. Preserve unselected/general receipts; never overwrite the collected export with the selected subset.
- Confirm the disabled GitLab branch remains Off. Do not enable it or send milestone emails solely to test a local ACORN release. If milestone delivery itself is in demo scope, test it separately with approved test recipients.
Record timestamps, receipt IDs, evidence of both Teams deliveries, export freshness, and PDF result using role aliases. Do not put credentials, participant addresses, or personal screenshots into shared acceptance records. This case is required for a live end-to-end demo, independent of INTAKE-01β09.
INTAKE-11 β Exercise the confirmation buttons in a real terminal
The runner also leaves an unapplied four-receipt batch in confirmation, including one personnel entry. In the original PowerShell window, inspect it and capture the index hashes:
$confirmationBatch = (Get-Content (Join-Path $intakeRoot 'evidence.json') -Raw | ConvertFrom-Json).confirmationBatch
Push-Location (Join-Path $intakeRoot 'confirmation')
& $env:ACORN_BIN --offline intake inspect --config intake.json --batch-id $confirmationBatch
$before = Get-ChildItem */index.json | Get-FileHash | Select-Object Path, Hash
& $env:ACORN_BIN --offline intake apply --config intake.json --batch-id $confirmationBatch
Run the last command again for each attempt, without --yes and without output redirection:
- Press Enter with No selected. Expect cancellation, nonzero exit, and unchanged indexes.
- Repeat with Escape, then Ctrl+C. Expect cancellation and unchanged indexes each time.
- Repeat, select Yes with Tab/arrows or
y, and confirm. Expect the reviewed changes and no duplicate entries on replay.
After each cancellation, compare hashes with $before (no output means unchanged):
Compare-Object $before (Get-ChildItem */index.json | Get-FileHash | Select-Object Path, Hash) -Property Path, Hash
After approving apply, exercise the personnel candidate using this command, repeating it for each cancellation and then approval:
& $env:ACORN_BIN --offline intake run --index watershed-sensing/index.json --interactive
No skips a candidate; Escape/Ctrl+C or declining the final write must leave the index unchanged and create no new backup. Capture fresh hashes before these attempts. Approve the candidate and final write to verify the retained archived source and exact-byte backup. --force and automated redirected-input rejection do not substitute for this interactive check. Return to the repository when finished:
Pop-Location
15. Completion and cleanup
CLEANUP-01 β Display the artifacts for review
Get-ChildItem -LiteralPath $env:ACORN_QA_ROOT -Recurse | Select-Object FullName, Length
Write-Host "Review failures before cleanup. Workspace: $env:ACORN_QA_ROOT" -ForegroundColor Cyan
CLEANUP-02 β Remove only this disposable run
Run only after screenshots and logs have been saved. This deletion cannot be undone.
$resolved = (Resolve-Path -LiteralPath $env:ACORN_QA_ROOT).Path
$allowedRoot = (Resolve-Path -LiteralPath "$env:ACORN_REPO\target\qa-acceptance").Path
if (-not $resolved.StartsWith($allowedRoot, [System.StringComparison]::OrdinalIgnoreCase)) {
throw "[FAIL] Refusing to delete an unexpected path: $resolved"
}
Remove-Item -LiteralPath $resolved -Recurse -Force
Write-Host "[PASS] Removed disposable acceptance workspace: $resolved" -ForegroundColor Green
Release acceptance recommendation
A release candidate is acceptable when:
- every applicable test in sections 1β8 and 13 passes;
- INTAKE-01β09 pass against the exact release candidate, including PDF content and visual inspection, whenever intake/reporting is in release scope;
- INTAKE-11 passes on an interactive terminal, and INTAKE-10 passes before claiming a live SharePoint-to-report demonstration;
- skipped PDF/live/interactive checks remain visible as outstanding; unsupported standalone general updates and direct SharePoint OAuth download are excluded explicitly or block a demo that requires them;
- PDF and PowerPoint artifacts pass visual inspection on at least one supported workstation;
- watch and TUI behavior pass on an interactive terminal build;
- connected tests pass in an environment authorized for network access;
- infrastructure tests pass only when those capabilities are part of the release scope;
- incomplete interfaces continue to fail clearly until they are implemented;
- no test writes outside the disposable workspace except the explicitly documented skill cache/clipboard and owner-supervised infrastructure actions;
- every failure has the test ID, version, operating system, terminal output, and retained artifact path.
Git branch, commit, merge-request, filter, and ignore selectors share the same source-resolution pipeline across check, format, export, and link. They should receive a short repository-specific smoke test when a release changes that pipeline. Merge-request tests require authenticated GitLab or GitHub context and belong with owner-supervised integration testing.