Skip to content

Commit a7cb5b6

Browse files
luisleo526claude
andcommitted
ta.sma: exact-order window sum for short lengths
Pine defines ta.sma(src, length) as the mean of the last `length` values, which makes it a pure function of its window. The incremental running sum (`running_sum += src - popped`) broke that: the emitted value became a function of every value ever accumulated, and the periodic exact re-summation at (bar_count & 255) == 0 only re-PHASED that drift rather than removing it. The failure is observable at the signal layer through the stochRSI shape k = ta.sma(stoch, 3); d = ta.sma(k, 3). On a saturating source the three k-windows feeding d are rotations of one multiset, so the s_{t-2} term cancels identically and 9 * (k_t - d_t) = 2*s_t + s_{t-1} - 2*s_{t-3} - s_{t-4} is EXACTLY ZERO whenever s_t == s_{t-3} and s_{t-1} == s_{t-4}. stochRSI hits that constantly because stoch saturates at exactly 100.0 / 0.0 (x/x -> 1.0, *100 exact in IEEE-754) every time rsi is its own 14-window extremum. ta.crossover / ta.crossunder are strict inequalities, so no correct implementation at any precision may fire at such a bar. Running-sum drift broke those ties at +-1 ULP and manufactured crossings the mathematics forbids, with a sign set by the resync phase rather than by the data. Windows of length <= 16 now re-sum the buffer every bar: exact-order deterministic, history-independent, and bounded at 16 additions per bar. Longer windows keep the amortised incremental path, where the chained-tie geometry does not arise. recompute() follows the same rule so a recomputed bar and a freshly computed one with an identical window agree bit-for-bit. Corpus (312 strategies): 3 tapes move, 0 regress. composite-kkb-margin-100-pct-01 unmatched 1 -> 0 ta-stochastic-rsi-cross-01 matched 1292 -> 1305 (96.6% -> 97.6%) ta-triple-sma-stack-latch-01 metrics unchanged (100% both) Tier distribution unchanged: excellent=307 strong=4 anomaly=1. tests/test_ta_sma_exact_order.cpp pins the invariant (red before, green after): equal windows -> bitwise equal output, and chained SMA(3) rotation ties stay exactly tied so no cross can fire. Co-Authored-By: Claude Opus 5 <[email protected]>
1 parent 304adee commit a7cb5b6

3 files changed

Lines changed: 250 additions & 5 deletions

File tree

‎src/ta_moving_averages.cpp‎

Lines changed: 44 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,31 @@ double RMA::compute(double src) {
7171

7272
// --- SMA ---
7373

74+
// Windows at or below this length re-sum the whole buffer every bar instead of
75+
// carrying an incremental running sum.
76+
//
77+
// A running sum (`running_sum += src - popped`) makes ta.sma HISTORY-DEPENDENT
78+
// to the last ULP: the emitted value at bar N is a function of every value ever
79+
// accumulated, and the periodic exact re-summation below only re-PHASES that
80+
// drift rather than removing it. For short chained windows the difference is
81+
// observable at the signal layer. The stochRSI shape
82+
// k = ta.sma(stoch, 3); d = ta.sma(k, 3)
83+
// sits on EXACT algebraic ties whenever the source plateaus: with
84+
// stoch = [a, b, x, a, b] the three k-windows are rotations of one multiset, so
85+
// 9*(k_t - d_t) = 2*s_t + s_{t-1} - 2*s_{t-3} - s_{t-4} = 0
86+
// EXACTLY, in infinite precision. ta.crossover/ta.crossunder are strict
87+
// inequalities, so no correct implementation at any precision may fire there.
88+
// Running-sum drift breaks those ties at +-1 ULP and manufactures crossings the
89+
// mathematics forbids, with a sign that depends on the resync phase rather than
90+
// on the data.
91+
//
92+
// Direct re-summation of the window is exact-order deterministic and
93+
// history-independent: identical windows always produce identical sums, so
94+
// algebraic ties survive as ties. Bounded to short windows so the per-bar cost
95+
// stays <= 16 additions instead of O(length); long windows keep the amortised
96+
// incremental path, where chained-tie geometry does not arise.
97+
static constexpr int kSmaDirectSumMaxLength = 16;
98+
7499
SMA::SMA(int length)
75100
: buffer(length), length(length), bar_count(0), running_sum(0.0) {}
76101

@@ -117,6 +142,14 @@ double SMA::compute(double src) {
117142
return running_sum / length;
118143
}
119144

145+
if (length <= kSmaDirectSumMaxLength) {
146+
// Exact-order, history-independent window sum (see the note above
147+
// kSmaDirectSumMaxLength): equal windows must produce equal sums so
148+
// algebraic ties in chained short SMAs survive as ties.
149+
running_sum = recalculate_exact_sum();
150+
return running_sum / length;
151+
}
152+
120153
if (!is_na(popped)) {
121154
running_sum -= popped;
122155
}
@@ -373,11 +406,17 @@ double SMA::recompute(double src) {
373406
double old_val = buffer[0];
374407
buffer.update_front(src);
375408

376-
if (!is_na(old_val)) {
377-
running_sum -= old_val;
378-
}
379-
if (!is_na(src)) {
380-
running_sum += src;
409+
if (length <= kSmaDirectSumMaxLength) {
410+
// Same exact-order window sum compute() uses, so a recompute()d bar and
411+
// a freshly computed one with an identical window agree bit-for-bit.
412+
running_sum = recalculate_exact_sum();
413+
} else {
414+
if (!is_na(old_val)) {
415+
running_sum -= old_val;
416+
}
417+
if (!is_na(src)) {
418+
running_sum += src;
419+
}
381420
}
382421

383422
if (bar_count < length) {

‎tests/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ set(TEST_SOURCES
1313
test_ta_indicators_extras
1414
test_kc
1515
test_ta_rma_warmup
16+
test_ta_sma_exact_order
1617
test_ta_na_source_rules
1718
test_matrix
1819
test_matrix_identity

‎tests/test_ta_sma_exact_order.cpp‎

Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
// test_ta_sma_exact_order — ta.sma must be a FUNCTION OF ITS WINDOW.
2+
//
3+
// Pine defines ta.sma(src, length) as the mean of the last `length` values.
4+
// That makes it a pure function of the window: two bars whose windows hold the
5+
// same values must emit the same number, to the last bit, no matter what the
6+
// series did before.
7+
//
8+
// An incremental running sum (`running_sum += src - popped`) breaks that. The
9+
// emitted value becomes a function of every value ever accumulated, and a
10+
// periodic exact re-summation only re-PHASES the drift instead of removing it.
11+
//
12+
// The failure is observable at the signal layer through the stochRSI shape
13+
//
14+
// k = ta.sma(stoch, 3)
15+
// d = ta.sma(k, 3)
16+
//
17+
// Whenever the source plateaus, the three k-windows feeding d are rotations of
18+
// a single multiset, so the s_{t-2} term cancels identically and
19+
//
20+
// 9 * (k_t - d_t) = 2*s_t + s_{t-1} - 2*s_{t-3} - s_{t-4}
21+
//
22+
// which is EXACTLY ZERO whenever s_t == s_{t-3} and s_{t-1} == s_{t-4}. stochRSI
23+
// hits that constantly: the stoch value saturates at exactly 100.0 or 0.0
24+
// (x/x -> 1.0 -> *100 is exact in IEEE-754) every time rsi is its own window
25+
// extremum. ta.crossover / ta.crossunder are STRICT inequalities, so at such a
26+
// bar no correct implementation at any precision may fire. Running-sum drift
27+
// breaks the tie at +-1 ULP and manufactures crossings the mathematics forbids.
28+
//
29+
// These tests pin the invariant directly: equal windows -> bitwise equal
30+
// output, and chained SMA(3) rotation ties stay exactly tied.
31+
32+
#include <pineforge/ta.hpp>
33+
#include <pineforge/na.hpp>
34+
35+
#include <cmath>
36+
#include <cstdio>
37+
#include <vector>
38+
39+
using namespace pineforge;
40+
41+
static int tests_passed = 0;
42+
static int tests_failed = 0;
43+
44+
static bool exact_eq(double a, double b) {
45+
if (is_na(a) && is_na(b)) return true;
46+
if (is_na(a) || is_na(b)) return false;
47+
return a == b; // bit-exact: ULP tolerance would hide the drift entirely
48+
}
49+
50+
#define CHECK_EXACT(actual, expected, tag) do { \
51+
double _a = (actual); \
52+
double _e = (expected); \
53+
if (exact_eq(_a, _e)) { tests_passed++; } \
54+
else { \
55+
tests_failed++; \
56+
std::printf("FAIL %s: got %.17g want %.17g (diff %.3g)\n", \
57+
(tag), _a, _e, _a - _e); \
58+
} \
59+
} while (0)
60+
61+
#define CHECK_TRUE(cond, tag) do { \
62+
if (cond) { tests_passed++; } \
63+
else { tests_failed++; std::printf("FAIL %s\n", (tag)); } \
64+
} while (0)
65+
66+
// ---------------------------------------------------------------------------
67+
// 1. Equal windows must produce bitwise-equal output regardless of history.
68+
//
69+
// Two SMA(3) instances are driven with DIFFERENT prefixes (one with values
70+
// large enough to shred the low bits of a running accumulator), then converged
71+
// onto the same final window. Pine semantics say they must agree exactly.
72+
// ---------------------------------------------------------------------------
73+
static void test_window_is_the_only_state() {
74+
ta::SMA a(3), b(3);
75+
76+
// Prefix A: quiet.
77+
for (int i = 0; i < 40; ++i) a.compute(1.0);
78+
// Prefix B: a huge value first, so an incremental sum carries absorption
79+
// error that a window re-sum would never produce.
80+
b.compute(1.0e16);
81+
for (int i = 0; i < 39; ++i) b.compute(1.0);
82+
83+
// Converge both onto the identical window {0.1, 0.2, 0.3}.
84+
double va = na<double>(), vb = na<double>();
85+
const double win[3] = {0.1, 0.2, 0.3};
86+
for (int i = 0; i < 3; ++i) { va = a.compute(win[i]); vb = b.compute(win[i]); }
87+
88+
CHECK_EXACT(va, vb, "SMA(3): identical windows, different history -> equal");
89+
// And it must equal a fresh instance fed only that window.
90+
ta::SMA c(3);
91+
double vc = na<double>();
92+
for (int i = 0; i < 3; ++i) vc = c.compute(win[i]);
93+
CHECK_EXACT(va, vc, "SMA(3): matches a freshly-seeded instance");
94+
}
95+
96+
// ---------------------------------------------------------------------------
97+
// 2. The stochRSI tie: chained SMA(3) over a saturating plateau.
98+
//
99+
// Feed a stoch-like series that saturates at exactly 100.0 with one arbitrary
100+
// unsaturated value threaded through, forming the rotation pattern
101+
// [.., a, b, x, a, b]
102+
// At the final bar 9*(k-d) == 0 exactly, so k must equal d BITWISE and neither
103+
// ta.crossover nor ta.crossunder may fire.
104+
// ---------------------------------------------------------------------------
105+
static void test_chained_sma3_rotation_tie_is_exact() {
106+
// A long saturated run first, so any running accumulator has drifted.
107+
std::vector<double> series;
108+
for (int i = 0; i < 300; ++i) series.push_back(100.0);
109+
// Rotation window: s[t-4]=100, s[t-3]=100, s[t-2]=x, s[t-1]=100, s[t]=100.
110+
const double x = 37.912345678901234;
111+
series.push_back(100.0);
112+
series.push_back(100.0);
113+
series.push_back(x);
114+
series.push_back(100.0);
115+
series.push_back(100.0);
116+
117+
ta::SMA k_sma(3), d_sma(3);
118+
double k = na<double>(), d = na<double>();
119+
for (double s : series) {
120+
k = k_sma.compute(s);
121+
d = d_sma.compute(k);
122+
}
123+
CHECK_EXACT(k, d, "stochRSI rotation tie: k == d bitwise");
124+
CHECK_TRUE(!(k > d), "rotation tie: ta.crossover must not fire (k > d false)");
125+
CHECK_TRUE(!(k < d), "rotation tie: ta.crossunder must not fire (k < d false)");
126+
127+
// Same shape on the 0.0 saturation side.
128+
std::vector<double> zseries;
129+
for (int i = 0; i < 300; ++i) zseries.push_back(0.0);
130+
zseries.push_back(0.0);
131+
zseries.push_back(0.0);
132+
zseries.push_back(x);
133+
zseries.push_back(0.0);
134+
zseries.push_back(0.0);
135+
136+
ta::SMA k0(3), d0(3);
137+
double kk = na<double>(), dd = na<double>();
138+
for (double s : zseries) { kk = k0.compute(s); dd = d0.compute(kk); }
139+
CHECK_EXACT(kk, dd, "stochRSI rotation tie (0.0 side): k == d bitwise");
140+
CHECK_TRUE(!(kk > dd) && !(kk < dd), "0.0-side tie: no cross may fire");
141+
}
142+
143+
// ---------------------------------------------------------------------------
144+
// 3. A fully saturated plateau must hold k == d == the plateau level exactly,
145+
// at EVERY bar of the plateau -- not merely at a resync boundary.
146+
//
147+
// This is the case that made the engine's 256-bar resync observable: the
148+
// spurious crossing appeared or vanished depending on where the plateau fell
149+
// relative to (bar_count & 255) == 0.
150+
// ---------------------------------------------------------------------------
151+
static void test_saturated_plateau_never_crosses() {
152+
ta::SMA k_sma(3), d_sma(3);
153+
double k = na<double>(), d = na<double>();
154+
int crossings = 0;
155+
// Long enough to sweep several resync phases (256-bar period).
156+
for (int i = 0; i < 1200; ++i) {
157+
// Alternate 0.0 / 100.0 plateaus long enough for k and d to settle.
158+
double s = ((i / 37) % 2 == 0) ? 100.0 : 0.0;
159+
double pk = k, pd = d;
160+
k = k_sma.compute(s);
161+
d = d_sma.compute(k);
162+
if (is_na(k) || is_na(d) || is_na(pk) || is_na(pd)) continue;
163+
// Inside a settled plateau (k, d and the previous pair all at the same
164+
// saturation level) no crossing may occur.
165+
bool settled = (k == d) && (pk == pd);
166+
if (settled) continue;
167+
// Count real sign changes; these are legitimate plateau transitions.
168+
if ((k > d && pk <= pd) || (d > k && pd <= pk)) crossings++;
169+
}
170+
// 1200 bars / 37-bar plateaus ~ 32 transitions; the exact count is not the
171+
// point -- the point is that it is FINITE and driven by transitions, not by
172+
// hundreds of ULP flips inside flat plateaus.
173+
CHECK_TRUE(crossings < 80,
174+
"saturated plateaus: crossings driven by transitions, not ULP noise");
175+
std::printf(" (plateau sweep: %d crossings over 1200 bars)\n", crossings);
176+
}
177+
178+
// ---------------------------------------------------------------------------
179+
// 4. Ordinary SMA values are unchanged: the fix must not move real arithmetic.
180+
// ---------------------------------------------------------------------------
181+
static void test_plain_sma_values_unchanged() {
182+
ta::SMA s(3);
183+
CHECK_TRUE(is_na(s.compute(1.0)), "SMA(3) bar0 na");
184+
CHECK_TRUE(is_na(s.compute(2.0)), "SMA(3) bar1 na");
185+
CHECK_EXACT(s.compute(3.0), 2.0, "SMA(3) seed = (1+2+3)/3");
186+
CHECK_EXACT(s.compute(4.0), 3.0, "SMA(3) = (2+3+4)/3");
187+
CHECK_EXACT(s.compute(5.0), 4.0, "SMA(3) = (3+4+5)/3");
188+
189+
// A long window still uses the incremental path; check it still tracks.
190+
ta::SMA big(50);
191+
double last = na<double>();
192+
for (int i = 1; i <= 50; ++i) last = big.compute((double)i);
193+
CHECK_EXACT(last, 25.5, "SMA(50) seed = mean(1..50)");
194+
}
195+
196+
int main() {
197+
test_window_is_the_only_state();
198+
test_chained_sma3_rotation_tie_is_exact();
199+
test_saturated_plateau_never_crosses();
200+
test_plain_sma_values_unchanged();
201+
202+
std::printf("test_ta_sma_exact_order: passed=%d failed=%d\n",
203+
tests_passed, tests_failed);
204+
return tests_failed == 0 ? 0 : 1;
205+
}

0 commit comments

Comments
 (0)