朱子纯 / xlyNewAi

…to file line 1; bench: simulate production anti-fab retry guard

- single ReAct entry for all free text: 7 fixed tools (useSkill + 6);
  flow knowledge lives in 5 skill files (resources/skills/*.md, first
  line = brief for prompt index; xly.skills.dir overrides for hot reload)
- useSkill returns skill body and logs skill_active; projection pins the
  active skill full text on the flow card until confirm/cancel/new skill
- system prompt from template prompts/system.txt (domain map + skill
  index); bench reads the same files to avoid replica drift
- delete IntentService/Intent/SlotFillService/LlmJsonClient/StateService/
  OkHttpUtil and route()/handleCreate/handleWrite/deriveWriteAction/
  ground/withState; dimension-split regex migrated into FormCollectTool
  prefill assist (model provably fails dimension splitting)
- anti-fabrication guards now unconditional (zero-tools+digits retry,
  WRITE_CLAIM check); invariant layer and button endpoints untouched
- bench_ext: --arch new (same-source prompt/skills) and --fifty
  (bench50 cases judged on full trajectory for both arches)

- chat:ledger becomes the only conversation store: model messages land as
  user/tool_call(payload)/tool_result/ai events via EventLogChatMemory
  (per-event rightPush, no more whole-blob read-modify-write races)
- EventProjectionService: one renderer for both frontend history and LLM
  context (system+flow card / deterministic digest of expired turns /
  ~6000-char verbatim recent turns, current turn payload-exact)
- pinned active-flow card (form pending / proposal pending / skill_active
  full text) unpinned on executed/cancelled/skill_done
- anti-fab retry nudges logged as internal user events (LLM-visible,
  hidden from history); grounded suffixes never duplicate user events
- delete ProjectedChatMemory; RedisChatMemoryStore demoted to legacy
  read-only compat (old convs age out via 30d TTL); conv list lazy cleanup
- tests: projection correctness + concurrent stream/confirm append safety

…on/form-continuation); old-arch baseline 9/12

resolveFk had no sBrandsId predicate, so a staged create/update could bind another
tenant's customer/product sId into this tenant's document (and doubled as a
cross-tenant existence oracle). Tables without a tenant column stay global.

docs/audit-agent-main-20260728.md is the external audit this and the previous
commit answer.

First tests in the repo — they pin the audit fixes that are easiest to silently
regress: no identity without introspection, no conversation across users, no
silent numeric corruption, no tenant-less FK query.

…p+CAS, tenant enforcement

Audit findings 1-5 confirmed real; root causes were (a) identity self-reported by
the client, (b) every failure path defaulting open.

- ERP /ai/whoami (saas-8s+ @c8e0057 follow-up) resolves token → real user/tenant/type;
  AuthzService.resolveIdentity is now the single identity entry point. Client-supplied
  userid/brandsid/usertype removed from all request bodies and from chat.html.
- dev-login is a real switch (erp.dev-login.enabled, default false) and its ERP creds
  no longer have built-in admin/666666 defaults; blank token in production → 401,
  never a silent fall back to the dev (sysadmin) account.
- conversations/op/form endpoints require login; conversation ids are namespaced by
  user id and ownership-checked (403 otherwise); /op/pending no longer returns sPayload.
- op confirm/cancel check the proposer, and confirm claims the draft via CAS so
  concurrent/repeat confirms cannot execute twice; bill numbers are regenerated at
  execution time instead of replaying the propose-time snapshot.
- FK options take the tenant from the introspected identity and return empty rather
  than dropping the sBrandsId predicate.
- secrets moved to env vars (DB_URL/DB_USERNAME/DB_PASSWORD/REDIS_*/LLM_*/ERP_BASEURL);
  allowMultiQueries=false. NOTE: the previously committed credentials must be rotated.
- ids interpolated into ERP URLs are validated (safeId) to stop query/path injection.
- update path rejects system columns, resolves FK names, and coerces by column type;
  numeric coercion now rejects unparseable input instead of writing 0/truncating, and
  proposal summaries show the value that will actually be written.
- ResponseStatusException keeps its 401/403 status instead of being wrapped as 200.
- anti-fabrication guard stays on when the intent gate itself fails.

作废/审核/修改 by bill number failed — locateRecord/lookupRecord always
filtered on the %Name field which can't match bill numbers.

- drop jQuery + page-in-page card; collapsible sidebar, single scroll area,
  light minimal style, markdown tables (markdown-it)
- collectForm: grid layout, required flags, typed controls, enum dropdown,
  FK secondary picker modal (multi-column list / search / paging / select)
- form submit posts structured fields to /api/agent/form/submit (no NL re-encoding);
  legacy FORM_SUBMIT_MARK route removed
- askUser card: option chips + always-available free-text input
- history replays from ledger endpoint; localStorage history dropped

…r free-input flag, readFormData paging

- POST /api/agent/form/submit: structured fields go straight to
  proposeWrite(action=create), no LLM re-encoding; ledger/state/memory recorded
- GET /api/agent/form/options: {columns,rows,total,page,pageSize} with name search
- askUser result carries allowFree=true (last option is always free text)
- readFormData(page?) with page shown in output

Verified exec-staging=true end-to-end against rebuilt local xlyEntry:
proposal → /ai/execStaging → DB value changed, status written back by ERP,
user-token identity enforced by @Authorization.

…ction, anti-fabrication guards

- LedgerService: append-only chat{conv} events incl. deterministic-path
  form/clarify/proposal/confirm-outcome; history replays from ledger
- StateService: chat:state:{conv} slots (上轮意图/最近实体/在办单据) written by code,
  fed to intent gate + write-slot extraction and appended to agent user text
- ProjectedChatMemory replaces MessageWindowChatMemory: full store + char-budget
  projection, old tool results collapsed to one line, current turn kept intact
- RedisChatMemoryStore.appendTurn patches memory holes on deterministic paths;
  op confirm/cancel outcomes recorded to ledger/state/memory
- anti-fabrication (tool_choice=required NOT enforced by Ollama 0.32.3, retested):
  query turn zero-tools+digits → one forced retry then flag; write claim without
  proposeWrite → corrective notice

- delete queryData (QueryTool, sqlChatModel bean, llm.sql-model, jsqlparser dep)
- delete kgSearch (findForms stays), delete loadSkill (SkillTool/SkillService)
- single system prompt (domain map + 业务常识) replaces 3 ToolScope versions; ToolScope removed
- intent gate reduced to 查询/新增/操作已有单据/其他 with class definitions only;
  write action derived from utterance in router
- dedup: shared locateRecord() in ProposeWriteTool; name-field/label lookups
  consolidated into FormResolverService

- Replace langchain4j-ollama with langchain4j-open-ai; AgentFactory/
  QueryTool depend on ChatModel/StreamingChatModel interfaces
- OllamaJsonClient -> LlmJsonClient: raw /v1/chat/completions with
  response_format json_schema; single OkHttpClient instance
- Thinking off at all 3 call sites via reasoning_effort=none (the only
  switch that works on /v1; think:false and /no_think are ignored)
- Drop client-side length params (num_ctx/num_predict/max_tokens) —
  server-side OLLAMA_CONTEXT_LENGTH=16384 on xlyllm covers them
- Unified tracing: TracingChatModelListener.record() shared by listener
  callbacks and LlmJsonClient; per-request model name from ctx; sql
  model now has the listener too (was untraced and mislabeled)
- Config keys langchain4j.ollama.* -> llm.{base-url,api-key,chat-model,
  sql-model}; both models = qwen3.6-27b-iq3:latest (tools+thinking+
  vision, verified: streaming tool-calls / json_schema / reasoning off)

Verified end-to-end on :8199: intent gate traced (340/28 tokens), agent
tool loop, correct answers; warm 1-4s, cold load ~96s (KEEP_ALIVE=30s).

P4 retirement + code slim-down (~18.5k lines deleted), on top of the
intent-gate WIP (docs §17/§18 + agent/tool refinements):

- Delete old 8-scene multi-agent stack: XlyErpService, SceneSelector/
  Chati/ErpAi/DynamicTableNl2Sql agents, DynamicToolProvider (62 meta
  tools), Scene/ToolMeta/ParamRule entities+mappers+startup caches
- Delete whole milvus/tts/ocr packages, /api/tts + /api/ocr endpoints,
  tts.html, python stream_server.py; strip dead TTS JS from chat.html
  (playByIndex/handleNormalResponse had no callers)
- Three-pass orphan sweep: 17 dead utils, 16 dead entities, dead
  constants/exceptions, RedisService+RedisConfig, duplicate
  JacksonConfig, OperableChatMemoryProvider, old prompt generators;
  fold PageController into MvcConfig; trim OkHttpUtil 495→39 lines
- pom: ~35→13 deps (drop mybatis/JPA/webflux/tika/pdfbox/poi/jieba/
  jsoup/gson/fastjson2/springdoc/mapstruct/pagehelper/pinyin4j/
  spring-retry/jnr-ffi/hutool/ocr/milvus/embeddings; add starter-jdbc);
  war 200M+→48M
- application.yml: drop milvus/tts/ocr/tesseract/mybatis blocks;
  GlobalExceptionHandler returns plain {code,message} JSON
- docs: mark P4 retirement done in agent-architecture §13/§18

Verified: mvn clean package OK; boot smoke on :8199 (Started 0.9s,
/chat 200, health db UP). Old /api/tts & /api/ocr now 404 by design.

Add a deterministic §5 intent stage before ReAct: extract {intent, form,
entities, missing slots} via constrained JSON, then expose only the 3-5
tools relevant to that intent instead of all 12.

New:
- agent/Intent, agent/ToolScope: intent + visible-tool-set model
- service/IntentService: constrained-JSON intent/entity extraction
- service/OllamaJsonClient: JSON-mode Ollama calls
- service/SlotFillService: slot extraction/prefill of known values

Wire through AgentFactory, SystemPromptService, QueryTool,
ProposeWriteTool, ErpClient, AgentChatController, OpController, chat.html.

Bug: '我要报价纸盒' -> agent put product '纸盒' into 客户名称, FK-lookup failed, then asked to CREATE a customer named 纸盒.

- System prompt rule 6: 新增报价 ALWAYS collectForm first; never map product/spec/盒型 to 客户名称;
  unknown customer -> let user pick from dropdown, never invent or offer to create.
- collectForm(entityKeyword, knownFieldsJson?): pre-fills the form from what the user already said
  (产品/数量/尺寸); customer/product render as dropdowns to pick real records.
- FK-not-found message no longer implies creating (was '请确认是否已存在' -> LLM read as 'create it?');
  now '不是系统里已有的X，请从下拉选真实存在的X，不要新建'.

Verified: '我要报价纸盒，1000个，长和宽50cm，高5cm' -> collectForm with 产品名称=纸盒(prefilled),
长/宽=50/高=5/数量=1000(prefilled), 客户名称=empty dropdown (no more guessing/create-customer).

User: form rendered every field as a plain text box; no type (dropdown/date/number), and
FK fields didn't pull options from their source tables.

- FormCollect schema now carries per-field type: fkselect | select | number | date | text
  (+ options for select, fkTable for fkselect, hint for placeholder).
  Types inferred from column data type for generic forms; curated for quote.
- New GET /api/agent/form/options?table=&q=&brandsid= -> FK options from the source table
  (elecustomer/eleproduct/elematerials...), whitelisted to tables that appear as sFkTable in the
  field dict (rejects arbitrary tables e.g. gdslogininfo), tenant-filtered + LIMIT.
- chat.html renderFormCollect renders by type: fkselect = searchable <input list=datalist> that
  fetches options on focus/typing; select = <select>; number = <input type=number>; date picker; text.
- Fix: tolerant label matching (strip parentheticals) so '多数量' maps whether or not the LLM keeps
  the '(逗号分隔)' hint -> multi-qty rows now reliably created.

Verified via agent: quote form shows 客户/产品/物料=fkselect, 单位/印刷/颜色/单双面=select,
尺寸/数量/单价/系数=number; options endpoint returns real names (客户 q=Little -> Little Antelope);
non-FK table rejected; full write incl 多数量 (BJD202607086 -> 2000/4000/6000).

User: chat form showed only 7 fields vs ERP's full 报价单. Root cause: 报价 is a cross-table
master-detail form; 印刷/颜色/单双面/部件/单价/系数/材料备注/多数量 live in slave tables, not the master.

- FormResolverService.curatedFields(quote): 17 fields tagged by target table (master/slave/note/manyqtys);
  印刷/颜色/单双面 have no column -> written as note into slave sMaterialsMemo.
- ProposeWriteTool.proposeQuote: routes fields to quoquotationmaster + quoquotationslave(印刷/部件, sParentId cascade) +
  quoquotationmanyqtys(多数量, one row per qty); FK name->id, type-coerce; emits __tables__ structured payload.
- ErpClient.createMulti + OpController: multi-table addUpdateDelBusinessData (master + slaves in one call).
- Pricing (核价) intentionally left to ERP (control/materials/process rows are engine-generated).

Verified via agent (real write): collectForm shows all 17 fields; created BJD202607084 = master(Little Antelope/妇科三醇乳膏/1.8mm双灰板/5000/12x8x5/3.5) + slave(只/部件30x20/系数1/材料备注含印刷=胶印;颜色=彩色;单双面=单面) + manyqtys(1000/3000/5000).

Root cause of the reported failure:
- collectForm sourced fields from gdsconfigformslave -> quote master's form config only exposes 3 system/header fields (失效时间/制单人/单据日期); user filled 单据日期(tCreateDate,datetime)='12' -> ERP type mismatch.
- proposeCreate never coerced by column type, never FK-resolved names, used addBusinessData (no billNo).

Fixes:
- FormResolverService: businessFields() curated authoritative label->{col,fk} for quoquotationmaster (客户名称/产品名称/数量/长/宽/高/单价 -> real writable cols sCustomerId/sProductId(FK)/dQty/dLength/dWidth/dHeight/dPrice), field-dict fallback for other tables; columnTypes/coerce/typeDefault/resolveFk/nextBillNo(BJD+YYYYMM+seq); isSystemColumn excludes reserve/money/rate/discount/person/date/bool/tenant/maker; resolveMasterForm excludes param/slave/control/config/color secondary tables.
- collectForm: fields from businessFields (excludes system/header); gdsconfigformslave fallback.
- proposeCreate: label->{col,fk} via businessFields; FK name->id; type-coerce; skip system cols; auto sId/sFormId + generated sBillNo; type-correct required defaults.
- ErpClient.createForm: addUpdateDelBusinessData(handleType=add)+moduleId (was addBusinessData).
- ProposeWriteTool.resolveForm unified to FormResolverService.

Verified via agent end-to-end (real write): collectForm shows correct fields; created quoquotationmaster BJD202607082 = Little Antelope / 妇科三醇乳膏 / qty5000 / dims / price, tenant+maker+date auto-injected.

…en, prod injects live token)

- ErpClient.execStaging(token, opId): POST /ai/execStaging/{opId} (user-token身份)
- OpController: erp.exec-staging.enabled -> delegate confirm to ERP executor; default false keeps tested direct path
- application-saaslocal.yml: erp.exec-staging.enabled=false (documented)

…at UI (question/form_collect/token)

- QueryTool: enforce table allowlist (viw_* + form data-sources + field-dict tables) via jsqlparser TablesNamesFinder -> blocks NL2SQL reading gdslogininfo/sysjurisdiction/ai_op_queue etc; single-table tenant predicate injection (sBrandsId); brand hint in prompt
- TracingChatModelListener: config-gated Langfuse ingestion export (generation span) via JDK HttpClient, no new deps; docker-compose.langfuse.yml self-host
- chat.html: send identity+token in chat req; render question(options) + form_collect(rich fields); forward Authorization on confirm/cancel
- application-saaslocal.yml: langfuse config (off by default)

…ormCollect + Skills + KgSearch

- AgentIdentity + AgentFactory: build agent per request with token+form-allowlist carried in tool instances (per-call context; robust vs LC4j callback threading)
- ErpClient: token-aware read/write/examine; user-token expiry does NOT silently re-login as dev-admin (no privilege escalation)
- AuthzService: devIdentity()/userIdentity() resolve granted-form set (sAuthsId)
- proposeExamine tool + OpController examine dispatch + forward user Authorization on confirm
- InteractionTool.askUser (structured options), FormCollectTool.collectForm (real gdsconfigformslave schema)
- SkillService + SkillTool.loadSkill + ai_skill digest in system prompt
- KgQueryTool.kgSearch: L2 neighbors/flow + L3 field->table
- tools ErpReadTool/ProposeWriteTool/QueryTool/FormCollectTool now per-request (not @Component)

…i_op_queue staging cols + ai_skill registry

- proposeCreate tool -> ai_op_queue draft(payload) -> confirm executes addBusinessData
- ai_op_queue: add sPayload/sSourceRef/bAutoExecute/sResultBillId/sErrorMsg/tExecutedDate/tExpireAt (create/examine/auto-flow ready)
- ai_skill registry table + seed playbooks (Skills §6)
- QueryTool: self-repair retry (feed SQL error back to model, up to 3x)
- TracingChatModelListener: per-call LLM latency/token/error trace (Langfuse hook point)

- AuthzService: form-level allowlist ported from backend getsAuthsIdNew (sysjurisdiction sKey set); sysadmin=all; wired into Read/lookupRecord/proposeUpdate/proposeDelete (admin dev-login => all-access; enforces per-user once prod passes through user token)
- proposeDelete + ErpClient.deleteForm (handleType=del) + OpController branches op type; HITL-gated like update
- docs/security-findings.md: report of the 5 current-system vulns (backend form-perm off, read-API-can-write, NL2SQL no-tenant/over-privilege/partial-SQL-safety/cache-skips-validation) for the business
- Verified end-to-end: read after authz = 93; propose-delete card -> confirm -> row actually deleted; audit rows written.

…nce) vs deferred (authz/create-delete-审核/FormCollect/staging-executor/Langfuse/NL2SQL)

- FormResolverService: shared KG resolution (entity->master table excl. viw_*, field 中文->column, name field, labels)
- ErpReadTool.lookupRecord(entity, record): reliably resolves the master table + returns a single record's full labelled fields — fixes the 'specific field of a specific record' case (e.g. 必胜客的销售员 -> 马艺祖)
- AuditService + ai_audit_log: immutable audit of propose/confirm/cancel/query (who/when/action/target/result)
- QueryTool.queryData(question): read-only SQL escape hatch — coder-model NL2SQL grounded on 字段字典, jsqlparser single-SELECT guard, blocks OUTFILE/LOAD_FILE/information_schema/SLEEP, forced LIMIT, audited. SAFE by construction; NL2SQL table/join accuracy for complex questions is limited (architecture-flagged, deferred).

Verified: lookupRecord works; audit rows written; Query guards + LIMIT + audit work (NL2SQL grounding still mis-picks entity master on multi-table questions).

- ai_op_queue: staging table for AI write ops (draft|confirmed|executed|failed|cancelled)
- ProposeWriteTool.proposeUpdate(entity, record, field, newValue): self-resolves the entity master table (excludes viw_* report views), resolves field via 字段字典, locates the unique record + old value, stages a DRAFT to ai_op_queue — DOES NOT execute; returns a proposal
- ErpClient.updateForm: executes via ERP addUpdateDelBusinessData with the frontend-compatible payload {data:[{sTable,name:master,column:[{handleType:update,sId,field:value}]}]}
- OpController: deterministic (non-LLM) POST /op/{id}/confirm (executes + writes back status) / cancel / GET pending
- AgentChatController.onToolExecuted: on proposeUpdate, attach conversation + push write_proposal SSE
- chat.html: confirm/cancel card + result echo
- system prompt: proposeUpdate flow; never claim a write is done before confirm

Verified end-to-end: '把必胜客简称改成必胜客中国' -> proposal card -> confirm -> ERP update -> DB changed; cancel -> DB unchanged. Nothing executes without explicit user confirm.

- ErpReadTool: optional keyword -> LIKE filter on the form's resolved name field (viw_kg_field_dict, *Name); keyword doc clarified (only for locating a named record, empty for counts)
- ErpClient.readForm: accepts filterField/filterValue -> bFilter
- KgQueryTool.findForms: rank by bAiTool(curated) + table + flow-connectivity + name length so the top hit is usually the canonical form
- SystemPromptService: hard 'always Simplified Chinese, no other language' rule (kills qwen language drift); prefer top form; read directly instead of over-asking
- AgentConfig: qwen3 think(false)/returnThinking(false) -> fast + disciplined tool-calling, no leaked chain-of-thought
- application-saaslocal.yml: chat model -> qwen3:14b

Verified: no more foreign-language leaks; '多少客户'->92; keyword lookup finds 必胜客; proc-backed stock forms readable; greeting 2s, form lookup ~6s. Known limit: which form/columns get picked for a specific-field-of-specific-record query still varies (deferred KgSearch).

- RedisChatMemoryStore (ChatMemoryStore): persist per-conversation messages to Redis (chat:mem:{convId}, 30d TTL) -> survives restart; agent memory now Redis-backed via MessageWindowChatMemory.builder().chatMemoryStore(...)
- ConversationService: per-user conversation metadata (chat:convs:{userId} hash), title from first message, list/create/delete/history
- ConversationController: GET/POST/DELETE /api/agent/conversations + GET /{id}/messages
- AgentChatController: touch conversation on each turn
- chat.html: conversation sidebar (list/switch/new/delete), load history on switch, refresh on send

Verified: multi-turn memory recalls facts; after full restart, conversation list + history + memory all persist (agent still recalls prior-turn facts).

- ErpClient: thin HTTP client to ERP xlyEntry; dev-login (/checklogin admin) mints+caches a real session token, auto re-login on session expiry (code=-2); reads getBusinessDataByFormcustomId (read-only params only, never bUpdate)
- ErpReadTool.readFormData(formId,moduleId): reads a form's real rows, renders Chinese column labels from viw_kg_field_dict, first 10 rows + total count
- wired into the agent (tools = findForms + readFormData); system prompt now describes the findForms->readFormData flow (still read-only, writes still 'under development')
- application-saaslocal.yml: erp.baseurl -> local xlyEntry :8697 + dev-login creds

Verified: '我们有多少客户' -> agent finds the form, reads 92 real customers, renders a clean Chinese table (name/category/salesperson/tax).

- ReActAgent: AiServices streaming tool-loop (replaces 8-scene SceneSelector path)
- AgentConfig: single agent = streaming Ollama(qwen2.5:14b) + KgQueryTool + per-conversation memory + L1-domain system prompt
- SystemPromptService: injects L1 domain map (viw_kg_domain) into system prompt; degrades gracefully if KG views missing
- KgQueryTool.findForms: read-only form-catalog lookup over local viw_ai_useful_forms (global metadata, no tenant) — interim formId resolver
- AgentChatController: POST /api/agent/chat SSE stream ({type:token|reset|done|error}); reset drops pre-tool-call narration
- chat.html: consume the new SSE endpoint (streamed), per-page conversationId, drop TTS-borrow path
- application-saaslocal.yml: add to worktree (was master-only untracked -> profile silently hit remote DB)

Verified running (saaslocal, :8099): streaming chat, tool loop returns real forms, write requests correctly report 'under development'.

…tions, standalone chat page, always user-present, designated model cloud/local, shared-DB multi-brand); backups recorded; formId deferred; refresh open-items

…rminal + reconcile on reconnect; manual stays sync