diff --git a/STATUS.md b/STATUS.md index 71dd443..cf7d9a2 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,6 +1,6 @@ # STATUS โ€” automotive-skills-suite -_Auto-generated 2026-06-03 by automotive-skills-daily-standup._ +_Auto-generated 2026-06-04 by automotive-skills-daily-standup._ | Builder | Domain | Paired Reviewer | Last Touched | Flag | |---------|--------|-----------------|--------------|------| @@ -86,5 +86,5 @@ _Auto-generated 2026-06-03 by automotive-skills-daily-standup._ - Builders: **76**, Reviewers: **76**, Paired ratio: **100.0%** - ๐ŸŸข Paired & fresh (โ‰ค30d): **2** ยท ๐ŸŸก Stale (>30d): **74** ยท ๐Ÿ”ด Orphan builders: **0** - Domain spread: safety=12, comms=8, quality=7, cyber=6, other=6, autosar=5, diagnostics=5, program-mgmt=5, v&v=5, aspice=4, sysml=4, calibration=3, mbse=3, sotif=3 -- Date: 2026-06-03 +- Date: 2026-06-04 - Source: scan of `skills/` directory + `git log -1` per file. diff --git a/docs/AUTONOMOUS_LOG.md b/docs/AUTONOMOUS_LOG.md index 0272d5e..3c55d10 100644 --- a/docs/AUTONOMOUS_LOG.md +++ b/docs/AUTONOMOUS_LOG.md @@ -404,3 +404,21 @@ Standout finding is non-DoD and more impactful than the trigger gaps: the SKILL. - Issue #17 needs human translation + triage call on Sun. - Tag-scheme ruling for Sat RELEASE (`v2026.06.W23` ISO-absolute vs `v2026.06.W1` per-month-spec-literal) still pending; defaulting to ISO-absolute unless instructed otherwise by Saturday. - Classifier-extraction target (#10) still inline-Python; today's STATUS regen needed the same alias map (item-definition โ†” item-def, ppap-package โ†” ppap) hand-maintained inline. + +## 2026-06-04 (autonomous run, POLISH) + +**Mode:** POLISH (Thursday โ€” third POLISH day of W23) +**Action:** W23 polish pass on fmeda-builder.skill (issue #15). Created new `docs/skill-polish-log/fmeda-builder.md` with first-pass audit (frontmatter, ASIL targets, 13-tab claim, workflow ordering, Common Pitfalls content, Key formulas section). Logged 2 medium-severity findings (Classification ladder has an unreachable "no mechanism" branch nested inside an "Else if allocated mechanism" block; "SMvDU (Safe Fault Metric)" non-standard acronym in pitfall #5) plus 3 low-severity items (SPFM/LFM intro inverts metric direction; `failure_mode_overrides.distribution_pct: 0.3` is likely 100ร— off; missing `!` punctuation on `#REF`/`#DIV/0` token names in Step 4 close). No `.skill` archive edits applied โ€” all findings touch math content or unit conventions, none on the narrow autonomous-edit allowlist (typo / over-length / missing-required-frontmatter-field). STATUS.md regenerated (date stamp only โ€” counts identical to yesterday). +**Files touched:** docs/skill-polish-log/fmeda-builder.md (new), STATUS.md (regenerated), docs/AUTONOMOUS_LOG.md (this entry) +**Tests:** N/A (no test suite in this repo yet) +**Skill count:** 76 builders / 76 reviewers / 100% paired +**Open issues:** 13 (unchanged composition โ€” #2 needs-human-triage, #3โ€“#10 carryovers, #11 ci, #12 W22 dfmea, #15/#16 W23 polish targets, #17 CN un-triaged) +**Notes:** Target-selection judgement call. Wed's follow-up bullet was explicit: "Thu POLISH should pick aspice-assessment-builder (#5) โ€” same shape as cs-concept (four-week carryover, description-quality label)." Picked fmeda-builder instead for two reasons. (1) Aspice-assessment was already polished once in W20 (2026-05-14 log) and Tue's W23 #1 cs-concept journal flagged the diminishing-returns pattern of repeat description-quality polishes on already-audited skills โ€” re-polishing aspice today would have produced the same finding shape (one missing trigger phrase) for the second time. (2) Fmeda had zero prior polish-log entries despite being on the W23 plan as target #4 (#15) AND despite being the 4-week-stale upstream artefact in the safety chain (TSC โ†’ FMEDA โ†’ Safety Case). A first-pass audit on a fresh target was likelier to surface real findings than a re-pass on a known one. The bet paid off โ€” the fmeda audit turned up two medium-severity findings on actual math/content (Classification ladder logic bug; "SMvDU" non-standard acronym) plus a 100ร— unit-convention discrepancy in the JSON example. None of these are description-quality items; they are real bugs in the FMEDA mentor content that a junior FuSa engineer reading the skill would propagate into a real submission. The Classification ladder finding in particular is reachable: an unmechanized failure mode read against the literal nested-if spec would be left unclassified rather than tagged SPF. Recommend the maintainer pull-request route rather than another polish-log appendix for fmeda โ€” the medium findings warrant real edits, not more flagging. +**Follow-ups:** +- Aspice-assessment-builder (#5) slips again โ€” now a likely W24 carryover. Recommend re-scoping the W24 PLAN slot from "another polish pass" to "open a maintainer PR closing #5 with the W20-drafted description rewrite already in `docs/skill-polish-log/aspice-assessment-builder.md`". The polish log has carried a ready-to-apply rewrite for three weeks; further polish passes are no longer adding information. +- Fmeda finding #1 (Classification ladder unreachable branch) is the highest-leverage maintainer item from this week's polish runs. Suggested flattened ladder in the polish log is drop-in. Same recommendation: maintainer PR, not another polish loop. +- Fmeda finding #2 (SMvDU acronym) needs a source-check before substitution โ€” best done by a human with ISO 26262-5:2018 Annex B in hand. Flag for maintainer. +- Fri DOCS day: weekly CHANGELOG roll-up will be lean (3 polish-log entries this week โ€” cs-concept, tara, fmeda; no `.skill` edits applied). Examples-stub coverage check should pick up `examples/fmeda-builder/README.md` since fmeda was touched but has no example stub. ESC BOM JSON shipped in the archive is a natural candidate to seed the example doc. +- Sat RELEASE tag-scheme ruling (`v2026.06.W23` ISO-absolute vs `v2026.06.W1` per-month) still pending after three flags. Defaulting to ISO-absolute on Saturday unless a maintainer flips it before then; the per-month spelling reads cleaner but breaks tag-sort continuity with the W20/W21/W22 series and is harder to map back to ISO week numbers in release notes. +- Issue #17 still untranslated; deferred to Sun TRIAGE. +- Classifier-extraction target (#10) still inline-Python; today's STATUS regen used the same alias map (item-definition โ†” item-def, ppap-package โ†” ppap) hand-maintained inline for the 7th consecutive run. diff --git a/docs/skill-polish-log/fmeda-builder.md b/docs/skill-polish-log/fmeda-builder.md new file mode 100644 index 0000000..89958d4 --- /dev/null +++ b/docs/skill-polish-log/fmeda-builder.md @@ -0,0 +1,204 @@ +# fmeda-builder polish log + +_Polish target for W23 (issue [#15](https://github.com/jherrodthomas/automotive-skills-suite/issues/15)). Reviewer: autonomous daily-standup task._ + +--- + +## 2026-06-04 โ€” first POLISH pass + +**Mode:** POLISH (Thursday โ€” third POLISH day of W23) +**File reviewed:** `skills/fmeda-builder.skill` (ZIP archive; SKILL.md is 9,673 bytes / ~155 lines; 10 files total in archive). +**DoD recap (from `docs/weekly/WEEK-2026-W23.md`):** +confirm the description's "13-tab workbook" claim matches the actual generator tab list, +spot-check the SPFM / LFM / PMHF target numbers against ISO 26262-5 Table 6, +and audit the "Key formulas" section in SKILL.md for internal consistency. + +### DoD verdict + +| DoD check | Result | +|---|---| +| 13-tab claim matches generator schema | **PASS** (cross-check vs. Output structure table; rows 00โ€“12 enumerated) | +| SPFM / LFM / PMHF target numbers vs. ISO 26262-5 Table 6 | **PASS** (90/97/99 for SPFM B/C/D, 60/80/90 for LFM B/C/D, 100/100/10 FIT for PMHF B/C/D โ€” all match standard) | +| "Key formulas" section internal consistency | **FAIL** โ€” Classification ladder has a logically unreachable branch (see finding #1) | + +### What's good + +- **Description is well-shaped.** 720 / 1024 chars, names every formal trigger phrase + a user would type ("FMEDA", "SPFM", "LFM", "PMHF", "ISO 26262-5", "diagnostic + coverage", "safety mechanisms"), and closes with the suite-standard "Always use this + skill instead of producing a freeform FMEDA in chat." line. No version-number gap of + the kind that bit `aspice-assessment-builder` (#5) and `cs-concept-builder` (#4) โ€” + there's only one ISO 26262 revision relevant here (2018). +- **Frontmatter is clean.** Both required keys present (`name`, `description`); YAML + parses without complaint; no drift fields. +- **ASIL targets are right.** Intro paragraph and PMHF pitfall section agree, and both + match ISO 26262-5:2018 Table 6 (SPFM โ‰ฅ90/97/99% for ASIL B/C/D; LFM โ‰ฅ60/80/90% for + B/C/D; PMHF โ‰ค100/100/10 FIT for B/C/D). This is the math users would most plausibly + mis-remember, so getting it locked in two places is a real strength. +- **13-tab list is exact.** The Output Structure table (00 Title Page โ†’ 12 References) + enumerates 13 rows and the Step 4 review checklist references tabs `03` through `11` + by name โ€” every reference resolves to a tab in the table. No drift between the + description's "13-tab" claim and the body. +- **Workflow ordering is correct.** Step 1 (BOM JSON) โ†’ Step 2 (read references, with + the right three files named in the right order) โ†’ Step 3 (run generator + recalc) + โ†’ Step 4 (analyst review, tab-by-tab). The Step 2 instruction to read the bundled + references is the pattern `aspice-assessment-builder` is *missing* and `fmeda-builder` + gets right. +- **Pre-requisites are explicit.** "A TSC workbook (produced by tsc-builder)" and + "A HW BOM JSON" โ€” names the upstream skill by ID, not by hand-wave. Makes the chain + dependency auditable. +- **Common pitfalls section is mentor-quality content.** Items #1 (FIT sourcing), + #2 (DC% evidence), #3 (residual fault accounting), and #4 (PMHF units) are exactly + the four mistakes a junior FuSa engineer makes in their first FMEDA. This is not + filler โ€” it's the right content for an audit-grade builder. + +### What to fix + +1. **MEDIUM โ€” Classification ladder in "Key formulas" has an unreachable branch.** + The current text reads (paraphrased): + + ``` + If not safety-related โ†’ S + If allocated mechanism with DC% and DC% = 100 โ†’ S + Else if allocated mechanism โ†’ check DC%: + If DC% < 100 โ†’ RF + If no mechanism or DC% = 0 โ†’ SPF + If allocated mechanism but covers multiple points โ†’ MPF_D or MPF_L + ``` + + The inner "If no mechanism or DC% = 0 โ†’ SPF" branch sits inside an "Else if + allocated mechanism" block โ€” so the "no mechanism" disjunct is logically + unreachable from that position. Two fixes are possible: (a) flatten the ladder so + "no mechanism โ†’ SPF" is a sibling branch of "allocated mechanism with DC% = 100 โ†’ + S", or (b) restate as a decision table. Either way, the current nesting cannot + be evaluated as written. Severity: **medium** (math content; a careful analyst + reading the formula spec literally will get the wrong classification for an + unmechanized failure mode). + +2. **MEDIUM โ€” "SMvDU (Safe Fault Metric)" acronym in pitfall #5 is not a standard + ISO 26262 term.** The Common Pitfalls section closes with: + > "This is why LFM has separate targets โ€” it's about SMvDU (Safe Fault Metric) + > paired with the architecture's diagnostic coverage of two-point faults." + + Neither ISO 26262-5:2018 nor the standard FuSa vocabulary (Annex B) defines + "SMvDU". The closest standard terms are ฮป_S (safe fault rate), MPF_DP (multipoint + fault, detectable/perceivable), and ฮป_S,LF (safe latent fault contribution). The + parenthetical "(Safe Fault Metric)" suggests the author meant either the Safe + Fault contribution to LFM, or possibly conflated MPF_L (latent multipoint) with + a Safe-Fault-style ratio. Severity: **medium** (a reader will Google "SMvDU" + and find nothing). Fix requires a real source check, not a typo correction. + +3. **LOW โ€” SPFM and LFM intro phrasings invert the metric direction.** Intro reads: + > "**SPFM** ... Coverage of *undetected* failure modes. Target: 90% (ASIL B), ..." + > "**LFM** ... Coverage of *latent* (undetected two-point) failures. Target: 60% ..." + + By ISO 26262-5 definition, SPFM = 1 โˆ’ ฮฃ(ฮป_SPF + ฮป_RF) / ฮฃ(ฮป_safety_related) and + LFM = 1 โˆ’ ฮฃ(ฮป_MPF,L) / (ฮฃ(ฮป_safety) โˆ’ ฮฃ(ฮป_SPF + ฮป_RF)). Both metrics measure the + fraction of safety-related faults that ARE handled (safe or detected), not the + fraction undetected. The intro's "Coverage of *undetected*" is the inverse of + what the formula actually computes โ€” a target of 99% SPFM means 99% of single- + point and residual fault rate is COVERED, not undetected. Severity: **low** + (the targets themselves are right and Step 4 later describes metrics correctly, + but the one-line definitions at the top of SKILL.md will mislead a reader + skimming the intro). + +4. **LOW โ€” `failure_mode_overrides` JSON example uses `distribution_pct: 0.3`.** + In the BOM JSON example the override row says `"distribution_pct": 0.3` but the + "Key formulas" section says `ฮป = FIT ร— (distribution % / 100)`. So a + distribution of 0.3 in the JSON would mean 0.3% of failures โ€” almost certainly + not the author's intent (a stuck-at fault is typically 30โ€“40% of an MCU's + failure mode budget, not 0.3%). Either the JSON example should read `30` or the + formula should read `ร— distribution_pct` (without the `/100`). Severity: **low** + (the example is documentary, not run-tested, but an analyst cloning the example + will produce an under-estimate by 100ร—). Fix needs a maintainer eyeball to + confirm the intended unit convention before flipping either side. + +5. **LOW (obs) โ€” Step 4 instruction "_When done, save + re-run `recalc.py`_" lacks + the same explicit error-checking the rest of the workflow does.** Other suite + skills (e.g. `tsc-builder`) name `#REF!` and `#DIV/0!` as the specific tokens to + grep for; this skill says "confirm all formulas evaluate and no #REF/#DIV/0 + errors remain." The phrasing is close but the `!` punctuation is dropped, which + matters for a CTRL-F. Severity: **low (obs)**. + +### Suggested edits (NOT applied this run) + +The autonomous-edit allowlist for the daily-standup task is narrow โ€” typo / +over-length description / missing-required-frontmatter-field. **None of the five +findings match that list:** + +- #1 (Classification ladder) needs a structural rewrite of a multi-line spec + block, not a surgical edit. Requires a 5โ€“10-line replacement with re-checked + precedence; better to do this with a human eyeball. +- #2 (SMvDU acronym) needs source research to identify the right replacement term, + not a substitution. Could be deletion, could be a re-naming to ฮป_S,LF โ€” either + way it's a content decision. +- #3 (SPFM/LFM intro inversion) is a two-sentence rewrite touching the + most-skimmed part of the SKILL.md. Worth doing carefully. +- #4 (distribution_pct unit) requires deciding which side (JSON or formula) is the + source of truth, then changing the other to match โ€” and possibly re-checking + the generator script against whichever side is kept. Not a surgical edit. +- #5 is a `!`-punctuation tidy. Borderline allowlist material but I'm declining + on the conservative reading. + +**No code edits committed today.** Issue #15 stays open with this log linked from +the journal entry. + +Proposed minimal rewrite of the SPFM / LFM intro lines (for human review): + +``` +- **SPFM** (Single-Point Failure Metric): Fraction of safety-related fault rate + that is *not* uncovered single-point or residual. Target: โ‰ฅ90% (ASIL B), + โ‰ฅ97% (C), โ‰ฅ99% (D). +- **LFM** (Latent Fault Metric): Fraction of remaining safety-related fault rate + (after SPFM) that is *not* uncovered latent multipoint. Target: โ‰ฅ60% (B), + โ‰ฅ80% (C), โ‰ฅ90% (D). +- **PMHF** (Probabilistic Metric for Hardware Failures): Raw uncovered failure + rate (ฮป_SPF + ฮป_RF), in FIT. Target: โ‰ค100 FIT (B/C), โ‰ค10 FIT (D). +``` + +Proposed flattened Classification ladder: + +``` +If not safety-related โ†’ S +Else if allocated mechanism AND DC% = 100 โ†’ S (safe due to detection) +Else if allocated mechanism AND covers multipoint: + If second-fault detection in place โ†’ MPF_D + Else โ†’ MPF_L +Else if allocated mechanism AND DC% < 100 โ†’ RF +Else (no mechanism OR DC% = 0) โ†’ SPF +``` + +### Other observations (not fixes, just notes for future passes) + +- The skill bundles three references (`methodology.md`, `failure_mode_libraries.md`, + `metrics_targets.md`) totaling ~23.6 KB โ€” substantial mentor content. Step 2 of + the workflow correctly tells Claude to read them. This is the right pattern; it + is what `aspice-assessment-builder` is missing. +- Generator script is 20.5 KB. Did not deep-audit script content this pass; the + STATUS regen + this log filled the slot. Worth a script-level pass on a future + POLISH day, particularly to confirm that the actual classification logic in + `generate_fmeda.py` matches whichever flattened ladder we land on for finding #1. +- No `.placeholder` cruft in this archive (unlike `aspice-assessment-builder`). + Archive is clean. +- The `examples/sample_fmeda_bom_esc.json` example is a real ESC (Electronic + Stability Control) BOM, not a toy. Likely valuable as a public reference + artefact for the eventual `examples/fmeda-builder/README.md` stub. + +### Severity roll-up + +| Finding | Severity | Action | +|---|---|---| +| Classification ladder has unreachable "no mechanism" branch | medium | flattened ladder drafted; await human review | +| "SMvDU (Safe Fault Metric)" non-standard acronym in pitfall #5 | medium | flagged; needs source research before substitution | +| SPFM/LFM intro lines invert metric direction ("Coverage of *undetected*") | low | three-line rewrite drafted; await human review | +| `failure_mode_overrides.distribution_pct: 0.3` likely 100ร— off | low | flagged; needs maintainer call on unit convention | +| `#REF/#DIV/0` lacks `!` punctuation in Step 4 close | low (obs) | flagged; suite-wide style item | + +**No code edits committed in this run.** Two medium-severity findings (logic bug +in Classification ladder + non-standard acronym) plus three low-severity items. +This is the **first** fmeda-builder polish pass โ€” earlier passes targeted +description-quality and trigger-list shape, but this run uncovered actual math / +content issues. Recommend the Classification ladder fix gets prioritized in a +maintainer touch-up rather than carried as another polish-log appendix; the logic +bug is reachable and would produce a wrong rating for an unmechanized failure +mode if read literally.