Proof of concept · synthetic B2B invoices · 6,468 invoices, 2023–2026
Every incoming invoice is checked before it is paid. Plain rules handle the clear cases, a model called Jev looks at the doubtful ones, and only what is still doubtful reaches a person. On this data, 99.6% of invoices are approved automatically and 21 go to a reviewer.
| Split | Invoices | Excluded | Rule flags | Sent to Jev | Jev flags | Approved | To review |
|---|---|---|---|---|---|---|---|
| tune | 5,115 | 5 | 11 | 329 | 5 | 5,094 | 16 |
| test | 1,208 | 2 | 3 | 68 | 1 | 1,202 | 4 |
| demo | 145 | 0 | 0 | 7 | 1 | 144 | 1 |
| All | 6,468 | 7 | 14 | 404 | 7 | 6,440 | 21 |
Invoices are split by billing date so nothing is tuned on the data it is judged on: tune before 2026-01-01 (choices are made here), test 2026-01-01 to 2026-08-31 (choices are checked here), demo from 2026-09-01 (the latest batch).
Arithmetic, prices, commissions, payment schedules and party data are checked line by line. The synthetic data is clean, so those rules all pass. The two rules about repeated billing do the real work.
| Rule | What it checks | Result |
|---|---|---|
line_arithmetic | Every line: package_quantity ≥ 1, unit price > 0, line amount = quantity × unit price | all pass |
invoice_total | Invoice total = Σ line amounts; line_count = number of lines; lines numbered 1..n | all pass |
commission | Each line's commission = 5% rounded half up; invoice commission = Σ line commissions | all pass |
price_list | Unit price = the product's list price for the year, as seen on earlier invoices; a new year's first price must be +0% … +10% per year above the last list price | all pass |
payment_schedule | Known schedule; installment count and first due date match it; 30 days apart; amounts sum to the total, differ by ≤ 1 cent, leftover cents first | all pass |
party_consistency | Invoice seller and business unit = the customer's; every line and installment matches the invoice header; each product exists, belongs to the invoice's business unit and keeps its catalogue name and category | all pass |
duplicate_lines | Same products and quantities as an invoice of the same customer ≤ 7 days earlier | all pass |
rebilled_products | Billed ≤ 2 days after the same customer's previous invoice with only products already on it (a partial rebill; Phase 4 found this rule catches them better than Jev) | 14 flagged |
near_duplicate_window | Billed ≤ 2 days after the same customer's previous invoice (Jev decides) | 418 to Jev |
Is this invoice the same order as the customer's previous invoice (billed again in whole or in part, or one order split across the two invoices), rather than a separate new order?
Jev (model jev-1.13.0) sees both invoices and the customer's history and answers yes or no with a probability. Scores ran from 0.05 to 0.46, median 0.19. Anything at or above 0.39 goes to a reviewer.
| Invoice | Split | Score | Outcome |
|---|---|---|---|
| 006328 | demo | 0.46 | Review |
| 002004 | tune | 0.43 | Review |
| 002737 | tune | 0.41 | Review |
| 000593 | tune | 0.40 | Review |
| 000752 | tune | 0.40 | Review |
| 002477 | tune | 0.39 | Review |
| 005663 | test | 0.39 | Review |
| 005577 | test | 0.38 | Approve |
| 000708 | tune | 0.37 | Approve |
| 003434 | tune | 0.36 | Approve |
| 006003 | test | 0.36 | Approve |
| 000892 | tune | 0.35 | Approve |
| 005883 | test | 0.34 | Approve |
| 000831 | tune | 0.33 | Approve |
| 003928 | tune | 0.33 | Approve |
| 002166 | tune | 0.32 | Approve |
| 002418 | tune | 0.32 | Approve |
| 004716 | tune | 0.32 | Approve |
| 005897 | test | 0.32 | Approve |
| 006158 | test | 0.32 | Approve |
| 001453 | tune | 0.31 | Approve |
| 002778 | tune | 0.31 | Approve |
| 001270 | tune | 0.30 | Approve |
| 005501 | test | 0.30 | Approve |
| 005656 | test | 0.30 | Approve |
| 006368 | demo | 0.30 | Approve |
| 000841 | tune | 0.29 | Approve |
| 001619 | tune | 0.29 | Approve |
| 002294 | tune | 0.29 | Approve |
| 002811 | tune | 0.29 | Approve |
| 003125 | tune | 0.29 | Approve |
| 004598 | tune | 0.29 | Approve |
| 006053 | test | 0.29 | Approve |
| 006178 | test | 0.29 | Approve |
| 006187 | test | 0.29 | Approve |
| 006191 | test | 0.29 | Approve |
| 000323 | tune | 0.28 | Approve |
| 000690 | tune | 0.28 | Approve |
| 001381 | tune | 0.28 | Approve |
| 004951 | tune | 0.28 | Approve |
| 004990 | tune | 0.28 | Approve |
| 001071 | tune | 0.27 | Approve |
| 001085 | tune | 0.27 | Approve |
| 003057 | tune | 0.27 | Approve |
| 003635 | tune | 0.27 | Approve |
| 003770 | tune | 0.27 | Approve |
| 004457 | tune | 0.27 | Approve |
| 005747 | test | 0.27 | Approve |
| 005832 | test | 0.27 | Approve |
| 006389 | demo | 0.27 | Approve |
| 006393 | demo | 0.27 | Approve |
| 000115 | tune | 0.26 | Approve |
| 000207 | tune | 0.26 | Approve |
| 000472 | tune | 0.26 | Approve |
| 001570 | tune | 0.26 | Approve |
| 002615 | tune | 0.26 | Approve |
| 002739 | tune | 0.26 | Approve |
| 003021 | tune | 0.26 | Approve |
| 003284 | tune | 0.26 | Approve |
| 003652 | tune | 0.26 | Approve |
| 004374 | tune | 0.26 | Approve |
| 000197 | tune | 0.25 | Approve |
| 001010 | tune | 0.25 | Approve |
| 001082 | tune | 0.25 | Approve |
| 001632 | tune | 0.25 | Approve |
| 001778 | tune | 0.25 | Approve |
| 002163 | tune | 0.25 | Approve |
| 002196 | tune | 0.25 | Approve |
| 002417 | tune | 0.25 | Approve |
| 002683 | tune | 0.25 | Approve |
| 003659 | tune | 0.25 | Approve |
| 003887 | tune | 0.25 | Approve |
| 004058 | tune | 0.25 | Approve |
| 004098 | tune | 0.25 | Approve |
| 004241 | tune | 0.25 | Approve |
| 004495 | tune | 0.25 | Approve |
| 004708 | tune | 0.25 | Approve |
| 005658 | test | 0.25 | Approve |
| 005793 | test | 0.25 | Approve |
| 005840 | test | 0.25 | Approve |
| 006423 | demo | 0.25 | Approve |
| 006458 | demo | 0.25 | Approve |
| 000367 | tune | 0.24 | Approve |
| 000381 | tune | 0.24 | Approve |
| 000586 | tune | 0.24 | Approve |
| 000988 | tune | 0.24 | Approve |
| 001051 | tune | 0.24 | Approve |
| 001136 | tune | 0.24 | Approve |
| 001159 | tune | 0.24 | Approve |
| 001233 | tune | 0.24 | Approve |
| 001598 | tune | 0.24 | Approve |
| 002532 | tune | 0.24 | Approve |
| 002698 | tune | 0.24 | Approve |
| 002754 | tune | 0.24 | Approve |
| 002982 | tune | 0.24 | Approve |
| 003142 | tune | 0.24 | Approve |
| 004104 | tune | 0.24 | Approve |
| 004458 | tune | 0.24 | Approve |
| 005845 | test | 0.24 | Approve |
| 006065 | test | 0.24 | Approve |
| 006235 | test | 0.24 | Approve |
| 000466 | tune | 0.23 | Approve |
| 000564 | tune | 0.23 | Approve |
| 000736 | tune | 0.23 | Approve |
| 000851 | tune | 0.23 | Approve |
| 001200 | tune | 0.23 | Approve |
| 001406 | tune | 0.23 | Approve |
| 001925 | tune | 0.23 | Approve |
| 002148 | tune | 0.23 | Approve |
| 002547 | tune | 0.23 | Approve |
| 002620 | tune | 0.23 | Approve |
| 003677 | tune | 0.23 | Approve |
| 004361 | tune | 0.23 | Approve |
| 004390 | tune | 0.23 | Approve |
| 004891 | tune | 0.23 | Approve |
| 004997 | tune | 0.23 | Approve |
| 005639 | test | 0.23 | Approve |
| 000274 | tune | 0.22 | Approve |
| 000289 | tune | 0.22 | Approve |
| 000498 | tune | 0.22 | Approve |
| 000504 | tune | 0.22 | Approve |
| 000547 | tune | 0.22 | Approve |
| 000622 | tune | 0.22 | Approve |
| 000655 | tune | 0.22 | Approve |
| 000845 | tune | 0.22 | Approve |
| 002162 | tune | 0.22 | Approve |
| 002314 | tune | 0.22 | Approve |
| 002337 | tune | 0.22 | Approve |
| 002475 | tune | 0.22 | Approve |
| 002684 | tune | 0.22 | Approve |
| 002875 | tune | 0.22 | Approve |
| 003769 | tune | 0.22 | Approve |
| 003921 | tune | 0.22 | Approve |
| 003986 | tune | 0.22 | Approve |
| 004156 | tune | 0.22 | Approve |
| 004904 | tune | 0.22 | Approve |
| 004989 | tune | 0.22 | Approve |
| 005238 | test | 0.22 | Approve |
| 005571 | test | 0.22 | Approve |
| 005711 | test | 0.22 | Approve |
| 000330 | tune | 0.21 | Approve |
| 000496 | tune | 0.21 | Approve |
| 000657 | tune | 0.21 | Approve |
| 000707 | tune | 0.21 | Approve |
| 000805 | tune | 0.21 | Approve |
| 000884 | tune | 0.21 | Approve |
| 000956 | tune | 0.21 | Approve |
| 001044 | tune | 0.21 | Approve |
| 001124 | tune | 0.21 | Approve |
| 001440 | tune | 0.21 | Approve |
| 001557 | tune | 0.21 | Approve |
| 001970 | tune | 0.21 | Approve |
| 002286 | tune | 0.21 | Approve |
| 002629 | tune | 0.21 | Approve |
| 002790 | tune | 0.21 | Approve |
| 002826 | tune | 0.21 | Approve |
| 003893 | tune | 0.21 | Approve |
| 004861 | tune | 0.21 | Approve |
| 004881 | tune | 0.21 | Approve |
| 005484 | test | 0.21 | Approve |
| 005509 | test | 0.21 | Approve |
| 005704 | test | 0.21 | Approve |
| 000793 | tune | 0.20 | Approve |
| 001047 | tune | 0.20 | Approve |
| 001523 | tune | 0.20 | Approve |
| 001574 | tune | 0.20 | Approve |
| 002015 | tune | 0.20 | Approve |
| 002060 | tune | 0.20 | Approve |
| 002567 | tune | 0.20 | Approve |
| 002924 | tune | 0.20 | Approve |
| 003145 | tune | 0.20 | Approve |
| 003315 | tune | 0.20 | Approve |
| 003905 | tune | 0.20 | Approve |
| 004030 | tune | 0.20 | Approve |
| 004319 | tune | 0.20 | Approve |
| 004404 | tune | 0.20 | Approve |
| 004814 | tune | 0.20 | Approve |
| 004943 | tune | 0.20 | Approve |
| 005260 | test | 0.20 | Approve |
| 000866 | tune | 0.19 | Approve |
| 001163 | tune | 0.19 | Approve |
| 001170 | tune | 0.19 | Approve |
| 001461 | tune | 0.19 | Approve |
| 001590 | tune | 0.19 | Approve |
| 001616 | tune | 0.19 | Approve |
| 001737 | tune | 0.19 | Approve |
| 002033 | tune | 0.19 | Approve |
| 002574 | tune | 0.19 | Approve |
| 002695 | tune | 0.19 | Approve |
| 002808 | tune | 0.19 | Approve |
| 002810 | tune | 0.19 | Approve |
| 002813 | tune | 0.19 | Approve |
| 003301 | tune | 0.19 | Approve |
| 003668 | tune | 0.19 | Approve |
| 003678 | tune | 0.19 | Approve |
| 003950 | tune | 0.19 | Approve |
| 004100 | tune | 0.19 | Approve |
| 004178 | tune | 0.19 | Approve |
| 004329 | tune | 0.19 | Approve |
| 004978 | tune | 0.19 | Approve |
| 005053 | tune | 0.19 | Approve |
| 005225 | test | 0.19 | Approve |
| 005404 | test | 0.19 | Approve |
| 005441 | test | 0.19 | Approve |
| 005654 | test | 0.19 | Approve |
| 005843 | test | 0.19 | Approve |
| 006052 | test | 0.19 | Approve |
| 006104 | test | 0.19 | Approve |
| 006119 | test | 0.19 | Approve |
| 006192 | test | 0.19 | Approve |
| 000226 | tune | 0.18 | Approve |
| 001008 | tune | 0.18 | Approve |
| 001228 | tune | 0.18 | Approve |
| 001488 | tune | 0.18 | Approve |
| 001621 | tune | 0.18 | Approve |
| 001924 | tune | 0.18 | Approve |
| 002067 | tune | 0.18 | Approve |
| 002277 | tune | 0.18 | Approve |
| 002748 | tune | 0.18 | Approve |
| 003850 | tune | 0.18 | Approve |
| 004086 | tune | 0.18 | Approve |
| 004424 | tune | 0.18 | Approve |
| 004499 | tune | 0.18 | Approve |
| 004661 | tune | 0.18 | Approve |
| 005701 | test | 0.18 | Approve |
| 005740 | test | 0.18 | Approve |
| 006054 | test | 0.18 | Approve |
| 006182 | test | 0.18 | Approve |
| 006237 | test | 0.18 | Approve |
| 000368 | tune | 0.17 | Approve |
| 000544 | tune | 0.17 | Approve |
| 000604 | tune | 0.17 | Approve |
| 000726 | tune | 0.17 | Approve |
| 001013 | tune | 0.17 | Approve |
| 001224 | tune | 0.17 | Approve |
| 001567 | tune | 0.17 | Approve |
| 001630 | tune | 0.17 | Approve |
| 001936 | tune | 0.17 | Approve |
| 002023 | tune | 0.17 | Approve |
| 002280 | tune | 0.17 | Approve |
| 002333 | tune | 0.17 | Approve |
| 002605 | tune | 0.17 | Approve |
| 003976 | tune | 0.17 | Approve |
| 004309 | tune | 0.17 | Approve |
| 004443 | tune | 0.17 | Approve |
| 004456 | tune | 0.17 | Approve |
| 004468 | tune | 0.17 | Approve |
| 005064 | tune | 0.17 | Approve |
| 005672 | test | 0.17 | Approve |
| 005754 | test | 0.17 | Approve |
| 005904 | test | 0.17 | Approve |
| 006011 | test | 0.17 | Approve |
| 006121 | test | 0.17 | Approve |
| 000022 | tune | 0.16 | Approve |
| 000331 | tune | 0.16 | Approve |
| 000982 | tune | 0.16 | Approve |
| 001350 | tune | 0.16 | Approve |
| 001466 | tune | 0.16 | Approve |
| 001511 | tune | 0.16 | Approve |
| 001556 | tune | 0.16 | Approve |
| 001629 | tune | 0.16 | Approve |
| 001920 | tune | 0.16 | Approve |
| 002554 | tune | 0.16 | Approve |
| 002885 | tune | 0.16 | Approve |
| 002948 | tune | 0.16 | Approve |
| 002979 | tune | 0.16 | Approve |
| 003076 | tune | 0.16 | Approve |
| 003205 | tune | 0.16 | Approve |
| 004270 | tune | 0.16 | Approve |
| 004797 | tune | 0.16 | Approve |
| 004946 | tune | 0.16 | Approve |
| 005795 | test | 0.16 | Approve |
| 006102 | test | 0.16 | Approve |
| 006233 | test | 0.16 | Approve |
| 000553 | tune | 0.15 | Approve |
| 001413 | tune | 0.15 | Approve |
| 002012 | tune | 0.15 | Approve |
| 003332 | tune | 0.15 | Approve |
| 003690 | tune | 0.15 | Approve |
| 004152 | tune | 0.15 | Approve |
| 004413 | tune | 0.15 | Approve |
| 004567 | tune | 0.15 | Approve |
| 004823 | tune | 0.15 | Approve |
| 004873 | tune | 0.15 | Approve |
| 004953 | tune | 0.15 | Approve |
| 005316 | test | 0.15 | Approve |
| 005652 | test | 0.15 | Approve |
| 006239 | test | 0.15 | Approve |
| 000122 | tune | 0.14 | Approve |
| 000813 | tune | 0.14 | Approve |
| 001379 | tune | 0.14 | Approve |
| 002032 | tune | 0.14 | Approve |
| 002421 | tune | 0.14 | Approve |
| 002798 | tune | 0.14 | Approve |
| 003723 | tune | 0.14 | Approve |
| 003930 | tune | 0.14 | Approve |
| 004711 | tune | 0.14 | Approve |
| 004959 | tune | 0.14 | Approve |
| 004993 | tune | 0.14 | Approve |
| 006064 | test | 0.14 | Approve |
| 001866 | tune | 0.13 | Approve |
| 002375 | tune | 0.13 | Approve |
| 002423 | tune | 0.13 | Approve |
| 003029 | tune | 0.13 | Approve |
| 005017 | tune | 0.13 | Approve |
| 005431 | test | 0.13 | Approve |
| 000558 | tune | 0.11 | Approve |
| 000747 | tune | 0.11 | Approve |
| 002624 | tune | 0.11 | Approve |
| 001605 | tune | 0.10 | Approve |
| 001981 | tune | 0.10 | Approve |
| 002213 | tune | 0.10 | Approve |
| 003984 | tune | 0.10 | Approve |
| 004387 | tune | 0.10 | Approve |
| 004444 | tune | 0.10 | Approve |
| 005069 | tune | 0.10 | Approve |
| 000592 | tune | 0.09 | Approve |
| 001405 | tune | 0.09 | Approve |
| 002043 | tune | 0.09 | Approve |
| 002510 | tune | 0.09 | Approve |
| 002897 | tune | 0.09 | Approve |
| 002953 | tune | 0.09 | Approve |
| 002998 | tune | 0.09 | Approve |
| 003442 | tune | 0.09 | Approve |
| 003783 | tune | 0.09 | Approve |
| 004957 | tune | 0.09 | Approve |
| 006215 | test | 0.09 | Approve |
| 000781 | tune | 0.08 | Approve |
| 000906 | tune | 0.08 | Approve |
| 000960 | tune | 0.08 | Approve |
| 000981 | tune | 0.08 | Approve |
| 000989 | tune | 0.08 | Approve |
| 001321 | tune | 0.08 | Approve |
| 001514 | tune | 0.08 | Approve |
| 001623 | tune | 0.08 | Approve |
| 002047 | tune | 0.08 | Approve |
| 002334 | tune | 0.08 | Approve |
| 002612 | tune | 0.08 | Approve |
| 002797 | tune | 0.08 | Approve |
| 002871 | tune | 0.08 | Approve |
| 003819 | tune | 0.08 | Approve |
| 004327 | tune | 0.08 | Approve |
| 004650 | tune | 0.08 | Approve |
| 005096 | tune | 0.08 | Approve |
| 005737 | test | 0.08 | Approve |
| 006028 | test | 0.08 | Approve |
| 006135 | test | 0.08 | Approve |
| 000238 | tune | 0.07 | Approve |
| 000396 | tune | 0.07 | Approve |
| 000514 | tune | 0.07 | Approve |
| 000682 | tune | 0.07 | Approve |
| 000694 | tune | 0.07 | Approve |
| 000842 | tune | 0.07 | Approve |
| 001050 | tune | 0.07 | Approve |
| 001091 | tune | 0.07 | Approve |
| 001171 | tune | 0.07 | Approve |
| 001452 | tune | 0.07 | Approve |
| 001848 | tune | 0.07 | Approve |
| 002008 | tune | 0.07 | Approve |
| 002216 | tune | 0.07 | Approve |
| 002678 | tune | 0.07 | Approve |
| 002872 | tune | 0.07 | Approve |
| 003008 | tune | 0.07 | Approve |
| 003605 | tune | 0.07 | Approve |
| 004274 | tune | 0.07 | Approve |
| 004303 | tune | 0.07 | Approve |
| 004419 | tune | 0.07 | Approve |
| 004742 | tune | 0.07 | Approve |
| 004785 | tune | 0.07 | Approve |
| 005071 | tune | 0.07 | Approve |
| 005121 | test | 0.07 | Approve |
| 005443 | test | 0.07 | Approve |
| 005930 | test | 0.07 | Approve |
| 006214 | test | 0.07 | Approve |
| 006223 | test | 0.07 | Approve |
| 006245 | test | 0.07 | Approve |
| 006455 | demo | 0.07 | Approve |
| 000108 | tune | 0.06 | Approve |
| 000346 | tune | 0.06 | Approve |
| 000952 | tune | 0.06 | Approve |
| 001020 | tune | 0.06 | Approve |
| 001232 | tune | 0.06 | Approve |
| 001371 | tune | 0.06 | Approve |
| 001617 | tune | 0.06 | Approve |
| 002322 | tune | 0.06 | Approve |
| 002580 | tune | 0.06 | Approve |
| 002639 | tune | 0.06 | Approve |
| 003177 | tune | 0.06 | Approve |
| 003689 | tune | 0.06 | Approve |
| 004101 | tune | 0.06 | Approve |
| 004247 | tune | 0.06 | Approve |
| 004364 | tune | 0.06 | Approve |
| 004899 | tune | 0.06 | Approve |
| 005097 | tune | 0.06 | Approve |
| 005333 | test | 0.06 | Approve |
| 005578 | test | 0.06 | Approve |
| 006025 | test | 0.06 | Approve |
| 000325 | tune | 0.05 | Approve |
| 000624 | tune | 0.05 | Approve |
| 000903 | tune | 0.05 | Approve |
| 001391 | tune | 0.05 | Approve |
| 002528 | tune | 0.05 | Approve |
| 004321 | tune | 0.05 | Approve |
| 004969 | tune | 0.05 | Approve |
The data has no answer key, so the evaluation plants cases with a known answer. Each one is built from real invoices: a rebill, a rebill with changed quantities, a split order, or a real new order moved next to the previous invoice. Then every case goes through the same rules and the same Jev question.
| Planted case | Should | Cases | Jev median score | Jev ≥ 0.39 | Rebill rule flags | Sent to review |
|---|---|---|---|---|---|---|
| Partial rebillSome products from the previous invoice billed again, same quantities. | Review | 30/30 | 0.51/0.48 | 100%/100% | 100%/100% | 100%/100% |
| Rebill, quantities changedProducts from the previous invoice billed again with different quantities. | Review | 30/30 | 0.35/0.33 | 40%/23.3% | 100%/100% | 100%/100% |
| Split orderOne real order divided over two invoices; every line is billed once. | Approve | 30/30 | 0.07/0.07 | 0%/0% | 0%/0% | 0%/0% |
| New order, moved closeA real later order of the same customer, moved to 1–2 days after the previous invoice. | Approve | 60/60 | 0.09/0.17 | 0%/3.3% | 1.7%/1.7% | 1.7%/3.3% |
Each cell shows tune / test. Green means the share is where it should be (≥ 90% for cases to review, ≤ 5% for cases to approve). “Sent to review” is the whole pipeline: rules first, then Jev.
Against real new orders its AUC is 1.00 on tune and 1.00 on test. Rebills with changed quantities are harder: only 40% / 23% score above the threshold.
“Billed ≤ 2 days later with only products already on the previous invoice” catches every planted rebill, both kinds, and flags 1.7% / 1.7% of real new orders. It now runs in tier 1, and Jev stays behind it as a second opinion.
Jev rates split orders as less alike than real new orders (AUC 0.23 / 0.18). That is acceptable: a split bills each line once, so it is approved. False alarms on new orders end at 1.7% / 3.3%.
| Signal | Rebill | Rebill, qty changed | Split |
|---|---|---|---|
| Jev (model) | 1.00/1.00 | 0.97/0.93 | 0.23/0.18 |
| Days since previous invoice | 0.49/0.49 | 0.51/0.51 | 0.51/0.51 |
| Products shared | 0.86/0.72 | 0.87/0.80 | 0.28/0.19 |
| Product overlap (Jaccard) | 0.94/0.90 | 0.96/0.94 | 0.28/0.19 |
| Shared products, same quantity | 1.00/0.98 | 0.49/0.47 | 0.49/0.47 |
| Total ÷ previous total | 0.10/0.06 | 0.05/0.05 | 0.34/0.61 |
| Same payment schedule | 0.58/0.61 | 0.58/0.61 | 0.58/0.61 |
| Both together match an earlier basket | 0.53/0.51 | 0.53/0.49 | 0.50/0.51 |
AUC of each signal at telling that kind apart from a real new order moved close (tune / test). 1.00 separates perfectly, 0.50 is a coin flip, below 0.50 points the wrong way.
The 21 invoices a person would look at, with the evidence each one arrives with.
| Invoice | Split | From | Previous | Days | Shared products | Overlap | Total ÷ prev | Why |
|---|---|---|---|---|---|---|---|---|
| 000345 | tune | Rule | 000333 | 1 | 3 | 0.60 | 1.51 | rebilled products: every product (P101, P107, P108) was already on 000333 (billed 2023-05-04) |
| 000593 | tune | Jev | 000586 | 1 | 5 | 0.71 | 0.98 | Jev rates it 0.40 likely the same order; billed 1 day after 000586 for the same customer |
| 000752 | tune | Jev | 000725 | 2 | 4 | 0.50 | 0.87 | Jev rates it 0.40 likely the same order; billed 2 days after 000725 for the same customer |
| 000940 | tune | Rule | 000917 | 2 | 3 | 0.50 | 0.93 | rebilled products: every product (P001, P002, P003) was already on 000917 (billed 2023-08-08) |
| 001005 | tune | Rule | 000982 | 2 | 1 | 0.33 | 1.13 | rebilled products: every product (P010) was already on 000982 (billed 2023-08-15) |
| 001269 | tune | Rule | 001260 | 1 | 4 | 0.67 | 0.72 | rebilled products: every product (P001, P002, P005, P007) was already on 001260 (billed 2023-10-16) |
| 001283 | tune | Rule | 001276 | 1 | 2 | 0.50 | 0.71 | rebilled products: every product (P003, P004) was already on 001276 (billed 2023-10-18) |
| 002004 | tune | Jev | 001988 | 1 | 3 | 0.75 | 1.00 | Jev rates it 0.43 likely the same order; billed 1 day after 001988 for the same customer |
| 002477 | tune | Jev | 002460 | 1 | 3 | 0.75 | 0.97 | Jev rates it 0.39 likely the same order; billed 1 day after 002460 for the same customer |
| 002626 | tune | Rule | 002595 | 2 | 1 | 0.50 | 0.81 | rebilled products: every product (P001) was already on 002595 (billed 2024-08-05) |
| 002737 | tune | Jev | 002726 | 1 | 2 | 0.67 | 1.14 | Jev rates it 0.41 likely the same order; billed 1 day after 002726 for the same customer |
| 002918 | tune | Rule | 002899 | 2 | 2 | 0.50 | 0.93 | rebilled products: every product (P005, P010) was already on 002899 (billed 2024-09-17) |
| 003002 | tune | Rule | 002996 | 2 | 3 | 0.50 | 0.79 | rebilled products: every product (P002, P005, P007) was already on 002996 (billed 2024-10-08) |
| 003649 | tune | Rule | 003635 | 1 | 3 | 0.50 | 0.63 | rebilled products: every product (P102, P105, P107) was already on 003635 (billed 2025-04-23) |
| 004088 | tune | Rule | 004073 | 1 | 2 | 0.50 | 0.96 | rebilled products: every product (P001, P007) was already on 004073 (billed 2025-07-02) |
| 004802 | tune | Rule | 004788 | 2 | 1 | 0.50 | 1.15 | rebilled products: every product (P110) was already on 004788 (billed 2025-10-28) |
| 005663 | test | Jev | 005648 | 1 | 2 | 0.29 | 0.54 | Jev rates it 0.39 likely the same order; billed 1 day after 005648 for the same customer |
| 005667 | test | Rule | 005641 | 2 | 3 | 0.60 | 1.28 | rebilled products: every product (P001, P005, P007) was already on 005641 (billed 2026-06-03) |
| 005670 | test | Rule | 005662 | 1 | 3 | 0.75 | 1.15 | rebilled products: every product (P002, P004, P010) was already on 005662 (billed 2026-06-04) |
| 005745 | test | Rule | 005732 | 1 | 1 | 0.33 | 0.95 | rebilled products: every product (P010) was already on 005732 (billed 2026-06-17) |
| 006328 | demo | Jev | 006312 | 1 | 4 | 0.67 | 0.87 | Jev rates it 0.46 likely the same order; billed 1 day after 006312 for the same customer |
The delivered files were audited against their documentation first, so the rules rest on facts about the data rather than assumptions.