auto(polish): W23 #3 fmeda-builder pass, regen STATUS, journal
Autonomous daily run (Thu W23). - Wrote first-pass polish log for fmeda-builder (issue #15): 2 medium-severity findings (unreachable Classification ladder branch; non-standard SMvDU acronym) + 3 low-severity items. No archive edits — all findings outside autonomous-edit allowlist. - Regenerated STATUS.md (date stamp only; 76/76 100% paired unchanged). - Picked fmeda over Wed-recommended aspice based on fresh-target vs. repeat-finding cost/benefit; bet paid off with real math/content findings. Generated by automotive-skills-daily-standup scheduled task.
This commit is contained in:
Automotive Skills Autonomous
@@ -1,6 +1,6 @@
|
||||
# STATUS — automotive-skills-suite
|
||||
|
||||
_Auto-generated 2026-06-03 by automotive-skills-daily-standup._
|
||||
_Auto-generated 2026-06-04 by automotive-skills-daily-standup._
|
||||
|
||||
| Builder | Domain | Paired Reviewer | Last Touched | Flag |
|
||||
|---------|--------|-----------------|--------------|------|
|
||||
@@ -86,5 +86,5 @@ _Auto-generated 2026-06-03 by automotive-skills-daily-standup._
|
||||
- Builders: **76**, Reviewers: **76**, Paired ratio: **100.0%**
|
||||
- 🟢 Paired & fresh (≤30d): **2** · 🟡 Stale (>30d): **74** · 🔴 Orphan builders: **0**
|
||||
- Domain spread: safety=12, comms=8, quality=7, cyber=6, other=6, autosar=5, diagnostics=5, program-mgmt=5, v&v=5, aspice=4, sysml=4, calibration=3, mbse=3, sotif=3
|
||||
- Date: 2026-06-03
|
||||
- Date: 2026-06-04
|
||||
- Source: scan of `skills/` directory + `git log -1` per file.
|
||||
|
||||
@@ -404,3 +404,21 @@ Standout finding is non-DoD and more impactful than the trigger gaps: the SKILL.
|
||||
- Issue #17 needs human translation + triage call on Sun.
|
||||
- Tag-scheme ruling for Sat RELEASE (`v2026.06.W23` ISO-absolute vs `v2026.06.W1` per-month-spec-literal) still pending; defaulting to ISO-absolute unless instructed otherwise by Saturday.
|
||||
- Classifier-extraction target (#10) still inline-Python; today's STATUS regen needed the same alias map (item-definition ↔ item-def, ppap-package ↔ ppap) hand-maintained inline.
|
||||
|
||||
## 2026-06-04 (autonomous run, POLISH)
|
||||
|
||||
**Mode:** POLISH (Thursday — third POLISH day of W23)
|
||||
**Action:** W23 polish pass on fmeda-builder.skill (issue #15). Created new `docs/skill-polish-log/fmeda-builder.md` with first-pass audit (frontmatter, ASIL targets, 13-tab claim, workflow ordering, Common Pitfalls content, Key formulas section). Logged 2 medium-severity findings (Classification ladder has an unreachable "no mechanism" branch nested inside an "Else if allocated mechanism" block; "SMvDU (Safe Fault Metric)" non-standard acronym in pitfall #5) plus 3 low-severity items (SPFM/LFM intro inverts metric direction; `failure_mode_overrides.distribution_pct: 0.3` is likely 100× off; missing `!` punctuation on `#REF`/`#DIV/0` token names in Step 4 close). No `.skill` archive edits applied — all findings touch math content or unit conventions, none on the narrow autonomous-edit allowlist (typo / over-length / missing-required-frontmatter-field). STATUS.md regenerated (date stamp only — counts identical to yesterday).
|
||||
**Files touched:** docs/skill-polish-log/fmeda-builder.md (new), STATUS.md (regenerated), docs/AUTONOMOUS_LOG.md (this entry)
|
||||
**Tests:** N/A (no test suite in this repo yet)
|
||||
**Skill count:** 76 builders / 76 reviewers / 100% paired
|
||||
**Open issues:** 13 (unchanged composition — #2 needs-human-triage, #3–#10 carryovers, #11 ci, #12 W22 dfmea, #15/#16 W23 polish targets, #17 CN un-triaged)
|
||||
**Notes:** Target-selection judgement call. Wed's follow-up bullet was explicit: "Thu POLISH should pick aspice-assessment-builder (#5) — same shape as cs-concept (four-week carryover, description-quality label)." Picked fmeda-builder instead for two reasons. (1) Aspice-assessment was already polished once in W20 (2026-05-14 log) and Tue's W23 #1 cs-concept journal flagged the diminishing-returns pattern of repeat description-quality polishes on already-audited skills — re-polishing aspice today would have produced the same finding shape (one missing trigger phrase) for the second time. (2) Fmeda had zero prior polish-log entries despite being on the W23 plan as target #4 (#15) AND despite being the 4-week-stale upstream artefact in the safety chain (TSC → FMEDA → Safety Case). A first-pass audit on a fresh target was likelier to surface real findings than a re-pass on a known one. The bet paid off — the fmeda audit turned up two medium-severity findings on actual math/content (Classification ladder logic bug; "SMvDU" non-standard acronym) plus a 100× unit-convention discrepancy in the JSON example. None of these are description-quality items; they are real bugs in the FMEDA mentor content that a junior FuSa engineer reading the skill would propagate into a real submission. The Classification ladder finding in particular is reachable: an unmechanized failure mode read against the literal nested-if spec would be left unclassified rather than tagged SPF. Recommend the maintainer pull-request route rather than another polish-log appendix for fmeda — the medium findings warrant real edits, not more flagging.
|
||||
**Follow-ups:**
|
||||
- Aspice-assessment-builder (#5) slips again — now a likely W24 carryover. Recommend re-scoping the W24 PLAN slot from "another polish pass" to "open a maintainer PR closing #5 with the W20-drafted description rewrite already in `docs/skill-polish-log/aspice-assessment-builder.md`". The polish log has carried a ready-to-apply rewrite for three weeks; further polish passes are no longer adding information.
|
||||
- Fmeda finding #1 (Classification ladder unreachable branch) is the highest-leverage maintainer item from this week's polish runs. Suggested flattened ladder in the polish log is drop-in. Same recommendation: maintainer PR, not another polish loop.
|
||||
- Fmeda finding #2 (SMvDU acronym) needs a source-check before substitution — best done by a human with ISO 26262-5:2018 Annex B in hand. Flag for maintainer.
|
||||
- Fri DOCS day: weekly CHANGELOG roll-up will be lean (3 polish-log entries this week — cs-concept, tara, fmeda; no `.skill` edits applied). Examples-stub coverage check should pick up `examples/fmeda-builder/README.md` since fmeda was touched but has no example stub. ESC BOM JSON shipped in the archive is a natural candidate to seed the example doc.
|
||||
- Sat RELEASE tag-scheme ruling (`v2026.06.W23` ISO-absolute vs `v2026.06.W1` per-month) still pending after three flags. Defaulting to ISO-absolute on Saturday unless a maintainer flips it before then; the per-month spelling reads cleaner but breaks tag-sort continuity with the W20/W21/W22 series and is harder to map back to ISO week numbers in release notes.
|
||||
- Issue #17 still untranslated; deferred to Sun TRIAGE.
|
||||
- Classifier-extraction target (#10) still inline-Python; today's STATUS regen used the same alias map (item-definition ↔ item-def, ppap-package ↔ ppap) hand-maintained inline for the 7th consecutive run.
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
# fmeda-builder polish log
|
||||
|
||||
_Polish target for W23 (issue [#15](https://github.com/jherrodthomas/automotive-skills-suite/issues/15)). Reviewer: autonomous daily-standup task._
|
||||
|
||||
---
|
||||
|
||||
## 2026-06-04 — first POLISH pass
|
||||
|
||||
**Mode:** POLISH (Thursday — third POLISH day of W23)
|
||||
**File reviewed:** `skills/fmeda-builder.skill` (ZIP archive; SKILL.md is 9,673 bytes / ~155 lines; 10 files total in archive).
|
||||
**DoD recap (from `docs/weekly/WEEK-2026-W23.md`):**
|
||||
confirm the description's "13-tab workbook" claim matches the actual generator tab list,
|
||||
spot-check the SPFM / LFM / PMHF target numbers against ISO 26262-5 Table 6,
|
||||
and audit the "Key formulas" section in SKILL.md for internal consistency.
|
||||
|
||||
### DoD verdict
|
||||
|
||||
| DoD check | Result |
|
||||
|---|---|
|
||||
| 13-tab claim matches generator schema | **PASS** (cross-check vs. Output structure table; rows 00–12 enumerated) |
|
||||
| SPFM / LFM / PMHF target numbers vs. ISO 26262-5 Table 6 | **PASS** (90/97/99 for SPFM B/C/D, 60/80/90 for LFM B/C/D, 100/100/10 FIT for PMHF B/C/D — all match standard) |
|
||||
| "Key formulas" section internal consistency | **FAIL** — Classification ladder has a logically unreachable branch (see finding #1) |
|
||||
|
||||
### What's good
|
||||
|
||||
- **Description is well-shaped.** 720 / 1024 chars, names every formal trigger phrase
|
||||
a user would type ("FMEDA", "SPFM", "LFM", "PMHF", "ISO 26262-5", "diagnostic
|
||||
coverage", "safety mechanisms"), and closes with the suite-standard "Always use this
|
||||
skill instead of producing a freeform FMEDA in chat." line. No version-number gap of
|
||||
the kind that bit `aspice-assessment-builder` (#5) and `cs-concept-builder` (#4) —
|
||||
there's only one ISO 26262 revision relevant here (2018).
|
||||
- **Frontmatter is clean.** Both required keys present (`name`, `description`); YAML
|
||||
parses without complaint; no drift fields.
|
||||
- **ASIL targets are right.** Intro paragraph and PMHF pitfall section agree, and both
|
||||
match ISO 26262-5:2018 Table 6 (SPFM ≥90/97/99% for ASIL B/C/D; LFM ≥60/80/90% for
|
||||
B/C/D; PMHF ≤100/100/10 FIT for B/C/D). This is the math users would most plausibly
|
||||
mis-remember, so getting it locked in two places is a real strength.
|
||||
- **13-tab list is exact.** The Output Structure table (00 Title Page → 12 References)
|
||||
enumerates 13 rows and the Step 4 review checklist references tabs `03` through `11`
|
||||
by name — every reference resolves to a tab in the table. No drift between the
|
||||
description's "13-tab" claim and the body.
|
||||
- **Workflow ordering is correct.** Step 1 (BOM JSON) → Step 2 (read references, with
|
||||
the right three files named in the right order) → Step 3 (run generator + recalc)
|
||||
→ Step 4 (analyst review, tab-by-tab). The Step 2 instruction to read the bundled
|
||||
references is the pattern `aspice-assessment-builder` is *missing* and `fmeda-builder`
|
||||
gets right.
|
||||
- **Pre-requisites are explicit.** "A TSC workbook (produced by tsc-builder)" and
|
||||
"A HW BOM JSON" — names the upstream skill by ID, not by hand-wave. Makes the chain
|
||||
dependency auditable.
|
||||
- **Common pitfalls section is mentor-quality content.** Items #1 (FIT sourcing),
|
||||
#2 (DC% evidence), #3 (residual fault accounting), and #4 (PMHF units) are exactly
|
||||
the four mistakes a junior FuSa engineer makes in their first FMEDA. This is not
|
||||
filler — it's the right content for an audit-grade builder.
|
||||
|
||||
### What to fix
|
||||
|
||||
1. **MEDIUM — Classification ladder in "Key formulas" has an unreachable branch.**
|
||||
The current text reads (paraphrased):
|
||||
|
||||
```
|
||||
If not safety-related → S
|
||||
If allocated mechanism with DC% and DC% = 100 → S
|
||||
Else if allocated mechanism → check DC%:
|
||||
If DC% < 100 → RF
|
||||
If no mechanism or DC% = 0 → SPF
|
||||
If allocated mechanism but covers multiple points → MPF_D or MPF_L
|
||||
```
|
||||
|
||||
The inner "If no mechanism or DC% = 0 → SPF" branch sits inside an "Else if
|
||||
allocated mechanism" block — so the "no mechanism" disjunct is logically
|
||||
unreachable from that position. Two fixes are possible: (a) flatten the ladder so
|
||||
"no mechanism → SPF" is a sibling branch of "allocated mechanism with DC% = 100 →
|
||||
S", or (b) restate as a decision table. Either way, the current nesting cannot
|
||||
be evaluated as written. Severity: **medium** (math content; a careful analyst
|
||||
reading the formula spec literally will get the wrong classification for an
|
||||
unmechanized failure mode).
|
||||
|
||||
2. **MEDIUM — "SMvDU (Safe Fault Metric)" acronym in pitfall #5 is not a standard
|
||||
ISO 26262 term.** The Common Pitfalls section closes with:
|
||||
> "This is why LFM has separate targets — it's about SMvDU (Safe Fault Metric)
|
||||
> paired with the architecture's diagnostic coverage of two-point faults."
|
||||
|
||||
Neither ISO 26262-5:2018 nor the standard FuSa vocabulary (Annex B) defines
|
||||
"SMvDU". The closest standard terms are λ_S (safe fault rate), MPF_DP (multipoint
|
||||
fault, detectable/perceivable), and λ_S,LF (safe latent fault contribution). The
|
||||
parenthetical "(Safe Fault Metric)" suggests the author meant either the Safe
|
||||
Fault contribution to LFM, or possibly conflated MPF_L (latent multipoint) with
|
||||
a Safe-Fault-style ratio. Severity: **medium** (a reader will Google "SMvDU"
|
||||
and find nothing). Fix requires a real source check, not a typo correction.
|
||||
|
||||
3. **LOW — SPFM and LFM intro phrasings invert the metric direction.** Intro reads:
|
||||
> "**SPFM** ... Coverage of *undetected* failure modes. Target: 90% (ASIL B), ..."
|
||||
> "**LFM** ... Coverage of *latent* (undetected two-point) failures. Target: 60% ..."
|
||||
|
||||
By ISO 26262-5 definition, SPFM = 1 − Σ(λ_SPF + λ_RF) / Σ(λ_safety_related) and
|
||||
LFM = 1 − Σ(λ_MPF,L) / (Σ(λ_safety) − Σ(λ_SPF + λ_RF)). Both metrics measure the
|
||||
fraction of safety-related faults that ARE handled (safe or detected), not the
|
||||
fraction undetected. The intro's "Coverage of *undetected*" is the inverse of
|
||||
what the formula actually computes — a target of 99% SPFM means 99% of single-
|
||||
point and residual fault rate is COVERED, not undetected. Severity: **low**
|
||||
(the targets themselves are right and Step 4 later describes metrics correctly,
|
||||
but the one-line definitions at the top of SKILL.md will mislead a reader
|
||||
skimming the intro).
|
||||
|
||||
4. **LOW — `failure_mode_overrides` JSON example uses `distribution_pct: 0.3`.**
|
||||
In the BOM JSON example the override row says `"distribution_pct": 0.3` but the
|
||||
"Key formulas" section says `λ = FIT × (distribution % / 100)`. So a
|
||||
distribution of 0.3 in the JSON would mean 0.3% of failures — almost certainly
|
||||
not the author's intent (a stuck-at fault is typically 30–40% of an MCU's
|
||||
failure mode budget, not 0.3%). Either the JSON example should read `30` or the
|
||||
formula should read `× distribution_pct` (without the `/100`). Severity: **low**
|
||||
(the example is documentary, not run-tested, but an analyst cloning the example
|
||||
will produce an under-estimate by 100×). Fix needs a maintainer eyeball to
|
||||
confirm the intended unit convention before flipping either side.
|
||||
|
||||
5. **LOW (obs) — Step 4 instruction "_When done, save + re-run `recalc.py`_" lacks
|
||||
the same explicit error-checking the rest of the workflow does.** Other suite
|
||||
skills (e.g. `tsc-builder`) name `#REF!` and `#DIV/0!` as the specific tokens to
|
||||
grep for; this skill says "confirm all formulas evaluate and no #REF/#DIV/0
|
||||
errors remain." The phrasing is close but the `!` punctuation is dropped, which
|
||||
matters for a CTRL-F. Severity: **low (obs)**.
|
||||
|
||||
### Suggested edits (NOT applied this run)
|
||||
|
||||
The autonomous-edit allowlist for the daily-standup task is narrow — typo /
|
||||
over-length description / missing-required-frontmatter-field. **None of the five
|
||||
findings match that list:**
|
||||
|
||||
- #1 (Classification ladder) needs a structural rewrite of a multi-line spec
|
||||
block, not a surgical edit. Requires a 5–10-line replacement with re-checked
|
||||
precedence; better to do this with a human eyeball.
|
||||
- #2 (SMvDU acronym) needs source research to identify the right replacement term,
|
||||
not a substitution. Could be deletion, could be a re-naming to λ_S,LF — either
|
||||
way it's a content decision.
|
||||
- #3 (SPFM/LFM intro inversion) is a two-sentence rewrite touching the
|
||||
most-skimmed part of the SKILL.md. Worth doing carefully.
|
||||
- #4 (distribution_pct unit) requires deciding which side (JSON or formula) is the
|
||||
source of truth, then changing the other to match — and possibly re-checking
|
||||
the generator script against whichever side is kept. Not a surgical edit.
|
||||
- #5 is a `!`-punctuation tidy. Borderline allowlist material but I'm declining
|
||||
on the conservative reading.
|
||||
|
||||
**No code edits committed today.** Issue #15 stays open with this log linked from
|
||||
the journal entry.
|
||||
|
||||
Proposed minimal rewrite of the SPFM / LFM intro lines (for human review):
|
||||
|
||||
```
|
||||
- **SPFM** (Single-Point Failure Metric): Fraction of safety-related fault rate
|
||||
that is *not* uncovered single-point or residual. Target: ≥90% (ASIL B),
|
||||
≥97% (C), ≥99% (D).
|
||||
- **LFM** (Latent Fault Metric): Fraction of remaining safety-related fault rate
|
||||
(after SPFM) that is *not* uncovered latent multipoint. Target: ≥60% (B),
|
||||
≥80% (C), ≥90% (D).
|
||||
- **PMHF** (Probabilistic Metric for Hardware Failures): Raw uncovered failure
|
||||
rate (λ_SPF + λ_RF), in FIT. Target: ≤100 FIT (B/C), ≤10 FIT (D).
|
||||
```
|
||||
|
||||
Proposed flattened Classification ladder:
|
||||
|
||||
```
|
||||
If not safety-related → S
|
||||
Else if allocated mechanism AND DC% = 100 → S (safe due to detection)
|
||||
Else if allocated mechanism AND covers multipoint:
|
||||
If second-fault detection in place → MPF_D
|
||||
Else → MPF_L
|
||||
Else if allocated mechanism AND DC% < 100 → RF
|
||||
Else (no mechanism OR DC% = 0) → SPF
|
||||
```
|
||||
|
||||
### Other observations (not fixes, just notes for future passes)
|
||||
|
||||
- The skill bundles three references (`methodology.md`, `failure_mode_libraries.md`,
|
||||
`metrics_targets.md`) totaling ~23.6 KB — substantial mentor content. Step 2 of
|
||||
the workflow correctly tells Claude to read them. This is the right pattern; it
|
||||
is what `aspice-assessment-builder` is missing.
|
||||
- Generator script is 20.5 KB. Did not deep-audit script content this pass; the
|
||||
STATUS regen + this log filled the slot. Worth a script-level pass on a future
|
||||
POLISH day, particularly to confirm that the actual classification logic in
|
||||
`generate_fmeda.py` matches whichever flattened ladder we land on for finding #1.
|
||||
- No `.placeholder` cruft in this archive (unlike `aspice-assessment-builder`).
|
||||
Archive is clean.
|
||||
- The `examples/sample_fmeda_bom_esc.json` example is a real ESC (Electronic
|
||||
Stability Control) BOM, not a toy. Likely valuable as a public reference
|
||||
artefact for the eventual `examples/fmeda-builder/README.md` stub.
|
||||
|
||||
### Severity roll-up
|
||||
|
||||
| Finding | Severity | Action |
|
||||
|---|---|---|
|
||||
| Classification ladder has unreachable "no mechanism" branch | medium | flattened ladder drafted; await human review |
|
||||
| "SMvDU (Safe Fault Metric)" non-standard acronym in pitfall #5 | medium | flagged; needs source research before substitution |
|
||||
| SPFM/LFM intro lines invert metric direction ("Coverage of *undetected*") | low | three-line rewrite drafted; await human review |
|
||||
| `failure_mode_overrides.distribution_pct: 0.3` likely 100× off | low | flagged; needs maintainer call on unit convention |
|
||||
| `#REF/#DIV/0` lacks `!` punctuation in Step 4 close | low (obs) | flagged; suite-wide style item |
|
||||
|
||||
**No code edits committed in this run.** Two medium-severity findings (logic bug
|
||||
in Classification ladder + non-standard acronym) plus three low-severity items.
|
||||
This is the **first** fmeda-builder polish pass — earlier passes targeted
|
||||
description-quality and trigger-list shape, but this run uncovered actual math /
|
||||
content issues. Recommend the Classification ladder fix gets prioritized in a
|
||||
maintainer touch-up rather than carried as another polish-log appendix; the logic
|
||||
bug is reachable and would produce a wrong rating for an unmechanized failure
|
||||
mode if read literally.
|
||||
Reference in New Issue
Block a user