AdsExecuteSQLDirect (and AdsExecuteSQL, which delegates to it) left the
result cursor positioned before the first row, where SAP ADS positions it
on the first row. A client that reads the current record immediately after
executing — the SAP-supported pattern, used by countless apps — got a
phantom empty row on every query. Proven with an aggregate, which returns
exactly one logical row: SELECT COUNT(*) came back as two rows (a blank,
then the count) on OpenADS versus one on SAP, and a plain SELECT yielded an
all-empty leading row.
Found by a differential parity dump of a real production dictionary (PMSYS)
against SAP ACE as the oracle. The fix positions the result on the first row at
the API boundary, so it applies uniformly to engine tables, memory-backed
system.*/aggregate results, and SQL-backend cursors, and to remote clients
(the server executes through the same entry point). AdsGotoTop is absolute,
so callers that already call it are unaffected — which is exactly why the whole
existing suite passed without catching this. Regression test added that reads
a result with no prior GotoTop.
Read-ahead was forward-only: a Skip(-1) browse (PgUp, or a report walking a
table in reverse) paid one round-trip per row. It now gets the same treatment
as a forward scan — a 300-row reverse scan over loopback dropped from 299 wire
requests to 7 (the depth ramp: 8+16+32+64×4).
- The server attaches a backward block (rows before the cursor, in
backward visit order) to a
Skipwith a negative step, walked and restored symmetrically to the forward block. - The prefetch lag counter is now signed (
cursor_lag = client_logical − server_cursor): a forward local drain moves it positive, a backward one negative, and the resyncstep + cursor_laghandles both with one formula. The forward path is byte-for-byte unchanged (the lag was always ≥ 0 there), which the existing forward tests confirm by still passing. - The depth ramp resets on a direction reversal, so a PgUp right after a long PgDn doesn't inherit the forward run's ceiling depth.
- New capability bit
kCapPrefetchBackward(0x04), and it is a correctness gate, not an optimization. A read-ahead block carries no direction marker on the wire, so a client that only understood forward blocks would drain a backward one the wrong way and serve the wrong record. The server therefore sends a backward block only to clients that advertise the bit. Safe in both mix directions: an old server ignores it; a new server never sends a backward block to a client that didn't ask for it (that client just keeps paying one round-trip per backward step).
A direction reversal itself costs one round-trip (the standing queue is on the wrong side and can't serve it); the wire skip refills the queue in the new direction. Verified end-to-end: a reverse scan is correct to the BoF boundary, mixed PgDn/PgUp navigation lands on the right recno, and the thrift test drops from 299 requests to 7 (and back to 299 with the backward path disabled, which is how the test proves the read-ahead is real).
Sequential prefetch shipped in M12.21, but it was explicitly disabled
whenever a controlling order was active: the look-ahead block was walked
on the engine cursor, which only ever moves in natural record order, so
for an ordered table it would have returned the wrong rows. That left the
browse that actually matters paying full price — Harbour's rddads
navigates on hOrdCurrent (an index handle) whenever an order is set, and
every such skip also re-sent a SetOrder frame first, then threw the
prefetch queue away. Two round-trips per row, cache discarded each time.
A 299-row ordered scan over loopback measured 598 wire requests before,
7 after (tests/unit/abi_remote_ordered_prefetch_test.cpp). No wire
format change.
- Index-order look-ahead.
pack_row_trailernow resolves an ordered table to its ABI handle — where the order and any scope live — so the block is walked in index order and the cursor restored afterwards. SetOrderis sent once per order change, not once per row. The newRemoteTable::server_order_idtracks what the server actually has installed (only aSetOrderack writes it), as distinct from the client's belief, which can be set with no round-trip at all via production-bag auto-open. That distinction is what makes skipping the frame safe.- Adaptive depth. Sequential detection lives on the server, as in OS
and DB read-ahead generally: depth ramps 8 → 16 → 32 → 64 per
consecutive forward
Skipon a table, and any reposition, write, or order change resets the run. A one-off "seek a record and read it" no longer drags a full 64-row block it will never look at. - Byte budget. The block is capped at 32 KB as well as by row count — 64 rows of a 4 KB-record table would otherwise be a 256 KB frame. SAP documents the same two-sided rule for its own client cache.
The caller's requested read-ahead depth now reaches the server, riding on
each Skip as an optional trailing [u16]. rddads already exposes the
call, so a Harbour app can use it directly.
0or1turns read-ahead off — SAP's documented remedy for a batch loop that edits most of the records it visits, where every write dumps the block anyway and reading ahead is pure waste. This previously had no way to be expressed.Nreads exactlyNrows per skip, overriding the automatic ramp (SAP's "aggressive" setting is 100). Capped at 512 so one request cannot become an unbounded server-side scan.
No capability bit, and no break in either version-mix direction. A new
client sending the extra 2 bytes to an old server is ignored by its length
check; an old client sending none to a new server reads as
kPrefetchDepthAuto. That sentinel is deliberately 0xFFFF and not 0,
because 0 already means "disable" — had absent-been-zero, every
pre-existing client would have silently lost read-ahead. That exact
regression is pinned by a raw-frame test
(tests/unit/network_skip_depth_test.cpp), which hand-builds a legacy
8-byte Skip because the ABI client can no longer produce one.
AdsSetRelation over a remote connection pointed the child at the wrong
parent record from the second skip of any scan onward. A parent/child grid
— every Mp10 relation browse — showed mismatched child data.
The cause: the relation read the parent's key with a wire GetField, which
reads the server's cursor. That cursor lags the client's logical position
by however many rows were served out of the read-ahead block, so the key was
one or more records behind. It read the parent's row into the client cache
first and then ignored it.
It hid because the only coverage skipped the parent exactly once, and the first skip of a scan always went to the wire (the block was still empty), leaving the two cursors coincidentally in sync. Every skip after it drained the block locally and drifted. Present since sequential prefetch landed (M12.21); found while adding the warm-GotoTop below, which makes even the first skip local and so exposed it immediately. The relation test now walks the whole parent instead of stopping at the first row.
Reading the cached row also removes a wire round-trip per parent row, per relation.
SeekAcknow carries the row it landed on. It used to be just[u8 found][u32 recno]— the client knew where it was but not what was there, so the firstAdsGetFieldafter a seek paid a second round-trip (FetchCurrentRow). Seek-then-read — the most common thing a business app does — cost 2 round-trips; it now costs 1. The relation code had been papering over this by firing aGotoRecordimmediately after every seek purely to pull the row down, so a parent browse with a child relation paid 2 round-trips per parent row; that frame is now only sent when talking to an older server.GotoTopnow comes back warm, carrying a read-ahead block with the row. A browse painting its first screen no longer pays a round-trip for the firstSkip.- Deliberately not
GotoBottom(a forward block past the last record is empty — that needs backward look-ahead, which is separate work) and deliberately notSeek/GotoRecord(relation navigation drives those once per parent row, and a block there would drag child rows over the wire every time for nothing). SAP draws the same line: read-ahead triggers on "a skip operation after ... any other movement operation" — the skip earns the block, not the movement.
Both are wire-compatible in either direction with no capability bit: an old
client requires size() >= 5 on a SeekAck and ignores trailing bytes, and
a new client against an old server sees the short ack, parses no trailer, and
falls back to the previous behaviour unchanged.
The central run-breaker that resets the adaptive read-ahead depth on a
reposition keys its map by table id, but Seek / SeekLast / SkipUnique /
SetScope / ClearScope frames lead with an index id. Passing that
through unresolved erased an absent key, so the table's ramp kept climbing
across a seek and the next Skip pulled a full ceiling block instead of
restarting at the floor — defeating the "seek a record, read it, move on"
case the ramp exists to protect. Rows were always correct (the block is
walked fresh), so this was a wasted-bandwidth bug, not wrong data. The
dispatcher now resolves the index id to its table before resetting.
Flipping AdsShowDeleted changes which rows are visible, but it did not drop
the client's read-ahead block — and that block holds rows already read ahead
of the cursor, under the old visibility. So a scan kept serving them: deleted
records continued to appear after SET DELETED ON (and live ones vanished after
SET DELETED OFF), with no wire traffic at all to hint anything was stale.
Measured on a 60-row table with the even records deleted: the scan returned 34
rows instead of 30, including deleted ones.
This sits exactly at the seam between remote SET DELETED (1.8.10/1.8.11) and
sequential prefetch (M12.21) — each is correct alone. AdsShowDeleted now
invalidates every remote table's cached row and look-ahead queue, and the server
ends the read-ahead run on all tables in the session (its payload carries no
table id, so it cannot go through the usual per-table run-breaker).
AdsSeek/AdsSeekLastinvalidated the cached row but not the look-ahead queue, so the nextSkip(1)could serve a stale pre-seek row with no wire traffic, and the following wire skip sent a step inflated by a lag that no longer applied.AdsSetIndexOrder/AdsSetIndexOrderByHandlehad the same hole: rows read in the previous order stayed queued.AdsGetRecorddid not settle the prefetch lag, so it returned the raw image of the record at the server's lagging cursor rather than the caller's current row.
bytes_outwas declared alongsidepackets_outbut never incremented, soAdsMgGetCommStats/sp_mgGetCommStatsalways reported 0 bytes sent. Now fed per reply frame.
pack_one_row_abire-resolved every column's name and type from the ABI on every row (three ABI calls per column, per row). Harmless at one row per ack; with a 64-row block it is thousands of calls perSkip. The schema of an open handle cannot change, so it is now resolved once per handle and cached.
AdsCacheRecordswas documented as a no-op because "OpenADS does not pre-cache rows" — untrue since M12.21. Now describes the automatic read-ahead and states plainly that the requested depth is still not honoured.docs/wire-protocol.md§5.8 documented a row-trailer layout that has never existed in the code. Replaced with the real format, including the look-ahead block and the consumed-lag protocol.
examples/fivewin/xbrowse_delscope.prg— FiveWin xBrowse over a remote table withSET DELETED ONand an index scope, records deleted inside the scoped range;/automode asserts no visible deleted row, no duplicate, and the exact expected key walk in both directions.
Follow-up to 1.8.10/1.8.11: with the deleted-row filter now actually active on the server, a remote natural-order browse showed a duplicate of the previous row where a deleted record used to appear (or silently truncated the walk a few rows early on tables longer than the prefetch window).
Root cause: Table::skip on the natural-order path computed the
landing as recno + delta (physical record arithmetic) and only
slid further while sitting on a hidden row. Every deleted/filtered
row strictly inside the range therefore left the cursor one visible
row short of where a Clipper SKIP n lands. Two remote mechanisms
turn that into user-visible corruption:
- The client prefetch fold (M12.21 option C) resynchronises the server
with
Skip(step + prefetch_consumed)— a multi-record skip. When it crossed a deleted row the server landed on the row the client had just painted and re-served it: the duplicated item. - The same short landing tripped the client's same-recno EOF heuristic, ending walks early on tables longer than the 64-row lookahead.
The bug was latent for years but unreachable remotely: before 1.8.10
the server always ran show_deleted=true, so physical and visible
arithmetic coincided. LOCAL apps could hit it with any dbSkip(n > 1)
over deleted rows or with AdsSetAOF-style filters.
Fix: when the deleted filter or an AOF is active, Table::skip now
walks one record at a time and counts only visible rows — same
semantics as the index-order path. The index path and the lookahead
restore (skip(-cursor_advance)) are unchanged and verified symmetric.
Requires updated openads_serverd (the engine runs server-side; the
DLL fix matters for LOCAL mode).
Files: engine/table.cpp.
Tests: local natural-order Skip(N) counts visible rows under SET DELETED ON, remote natural-order walk under SET DELETED ON has no duplicate rows, and remote natural-order walk longer than prefetch window under SET DELETED ON in abi_remote_index_nav_test.cpp (the
long-walk case reproduced the truncation: 66 rows seen instead of 69
pre-fix).
Follow-up to 1.8.10 (M12.31): a scoped remote browse still showed
deleted rows when the application ran SET DELETED ON before
AdsConnect60 — the startup order virtually every rddads / FiveWin
app uses (SET DELETED ON in Main, then connect). LOCAL mode was
unaffected.
Root cause, two gaps left by M12.31:
- Client: the
AdsShowDeletedbroadcast only notified remote connections open at call time. A connection created afterwards never received the flag, andconnect_with_transportdid not sync it. - Server: the
ShowDeletedhandler applied the flag toabi_conn_only if it already existed. That connection is created lazily (firstSetScope/SetOrder), so when the opcode arrived earlier the lazily-created ABI connection started with the defaultshow_deleted=true— and the per-connection flag overrides the engine global on ordered/scoped walks.
Fix:
RemoteConnection::connect_with_transportpushesShowDeleted(0)right afterConnectAckwhenever the client state is "hide deleted" (the server default is "show", so only that direction needs syncing).Sessionremembers the lastShowDeletedstate and re-applies it whenensure_abi_conncreates the lazy ABI connection.
Requires updated openace64.dll and openads_serverd. A 1.8.10
server honours the flag only if it changes while connected; pre-1.8.10
servers ignore it entirely.
Files: client.cpp, session.{h,cpp}.
Tests: remote scoped walk honours SET DELETED ON issued before connect and … before first ordered op in
abi_remote_index_nav_test.cpp. New wire probe
tools/remote_deleted_probe.cpp (openads_remote_deleted_probe)
reproduced the leak against a live pre-fix server and verified the fix
end-to-end over TCP.
With SET DELETED ON (AdsShowDeleted(0)), a scoped browse over a
remote alias (OrdScope / AdsSetScope + GotoTop/Skip) still
returned rows flagged deleted. LOCAL mode filtered them correctly.
Root cause: AdsShowDeleted only updated the client process; the server
session (sess_conn_ and the parallel ABI connection used for ordered
navigation) kept show_deleted=true, so index walks on the server
included deleted keys inside the active scope.
Fix:
- New wire opcodes
ShowDeleted/ShowDeletedAck(0xDA/0xDB).AdsShowDeletedpushes the flag to every openRemoteConnection. - Server handler applies it to the global engine flag,
sess_conn_, andabi_conn_(the handle used forordered_tables_navigation). SetScopehandler also callsAdsSetIndexOrderByHandleon the ABI table so scope stays bound to the active order without a priorSetOrderwire op.- Client releases
s.mubefore theShowDeletedwire round-trip (same deadlock pattern as remoteAdsOpenTable/AdsOpenIndex).
Requires updated openace64.dll and openads_serverd. Pre-M12.31
servers ignore the new opcode (best-effort); upgrade both sides together.
Files: wire.h, client.{h,cpp}, session.cpp, ace_exports.cpp.
Test: remote AdsSetScope with SET DELETED ON skips deleted rows in
abi_remote_index_nav_test.cpp.
The second mechanism from the v1.8.6/v1.8.7 regression report: a
Harbour rddads app has no way to call AdsSetCollation — its only OEM
signal is usCharType = ADS_OEM on AdsOpenTable (what
AdsSetCharType(ADS_OEM) produces) — and OpenADS ignored that
parameter. INDEX ON therefore built keys with UTF-8 casing and binary
sort while the app seeks with Harbour's CP-852 Upper() keys →
not-found on every row with Polish letters.
OpenADS now mirrors SAP's model (help: "Advantage Local Server
Configuration" / "Avoiding OEM Collation Mismatch Errors"): the OEM
collation language is a machine-level default, and tables opened
ADS_OEM pick it up with zero per-connection code.
AdsOpenTable/AdsCreateTablehonourusCharType;AdsGetTableCharTypereports the stored value (was hard-coded ANSI).- Default OEM collation configured via SAP-style
adslocal.cfg([SETTINGS]/OEM_CHAR_SET=NTXPL852, file next toopenace64.dllor in the current directory) or theOPENADS_OEM_COLLATIONenvironment variable (env wins;AdsSetCollationremains the per-connection override). - Effective collation resolved per table: CDX tags of ADS_OEM tables
build/compare/seek with the PL852 sort weights;
UPPER()in index expressions cases per table (thread-local evaluation scope), so ANSI tables keep UTF-8 case promotion untouched. - Unsupported
OEM_CHAR_SETvalues (USA, MAZOVIA, …) leave raw byte order, like SAP's shippingUSAdefault (#127 tracks more tables). - If OEM data was indexed on v1.8.6/v1.8.7,
REINDEXonce after configuring the collation (same rule as SAP after an OEM collation-language change).
v1.8.6 regression, reported with a clean 1.8.4→1.8.6 bisect: DbSeek on
a character tag under NTXPL852 returned not-found for existing records —
but only after the bag was closed and reopened (production usage), which
the test suite never exercised.
Cause was a two-commit interaction: v1.8.5 marked ANY reopened CDX tag
with an 8-byte key as FoxNumeric (a plain C(8) character tag included),
and v1.8.6 made that flag select the B+tree comparator (memcmp for
numeric encodings). A mis-marked character tag then descended a
collation-ordered tree in raw byte order and missed keys whose PL852
weight order differs from byte order (e.g. Ł = 0x9D).
The reopen path now decides the encoding from the key expression — bare
fields by schema type, computed expressions only when the key is 8 bytes
AND the expression provably evaluates numeric (Val() heuristic +
record-1 probe, same rules as index creation). The #130 numeric Val()
fix is preserved. New regression test closes and reopens the bag before
seeking.
Scoped browses over rddads (grids restricted to one parent key) returned either every FOR-matching row or no rows at all. Four root causes, all fixed:
ADS_TOP/ADS_BOTTOMconstants (ace.h): OpenADS used 0/1; the ACE convention (and Harbour'sads.ch) is 1/2.AdsSetScopetherefore never set the TOP bound for rddads callers — both bounds landed on BOTTOM. Note this also applies over the wire:openads_serverdmust be rebuilt together with the client, or TOP/BOTTOM invert silently.AdsGetKeyTypereturned the wrong constant family: it reported the seek/scope buffer encodings (ADS_STRINGKEY=1 /ADS_DOUBLEKEY=2) instead of the ACE field-type constants (ADS_STRING=4,ADS_NUMERIC=2,ADS_DATE=3,ADS_LOGICAL=1). rddads switchesOrdScope()'s key encoding on this value, read 1 asADS_LOGICAL, and sent a 1-byte"T"/"F"scope instead of the real key for every character-key tag. Now answers from the table schema for bare-field keys and falls back to the key encoding for computed expressions.$("contains") operator was a no-op in FOR-condition evaluation (STATUS $ 'IEC'passed every non-blank row). Real containment semantics implemented.AdsGetKeyCountignored the active scope on CDX orders — it returned the whole conditional index size. Now walks only the scoped range.
Also new, enabled by the AdsGetKeyType fix: rddads sends date scopes
and DbSeek(date) as julian-day doubles — AdsSetScope and AdsSeek
now convert those to the YYYYMMDD text key form for DBF date keys
(ADT/ADI date keys keep their packed binary path).
Verified end-to-end locally and over the wire against a live
openads_serverd (new gated tests: OPENADS_TEST_REMOTE scope walk /
key count, plus a seed helper via OPENADS_SEED_DIR). Trilingual
AdsGetKeyType reference pages updated.
- Removed a dead CP-852 upper table in
oem_collation.cppthat broke clang-Werrorbuilds (-Wunused-const-variable).
- Numeric
Val()DbSeekfix (#130):memcmpordering for FoxNumeric / NtxNumeric key comparison in CDX.
- OEM UPPER for NTXPL852/PL852 index keys + complete
Val()numeric support (#130); collation scoping fix (OEM UPPER only applies when an OEM collation is actually active).
- Single-pass batching for sibling tag rebuilds using the new
Table::collect_keys_for_multiple_expressionsprototype. When multiple tags in a bag need a full rebuild (common after ORDLSTCLEAR + repeatedINDEX ON), we now do one DBF scan instead of N. - Fast paths + OEM upper-casing for
UPPER(barefield)and similar common expressions (skips per-row Parser + UTF-8 work; uses proper CP-852 upper table for NTXPL852/PL852 data). - Added
oem_upper/lookup_oem_upper_tablesupport in oem_collation. - More unit test coverage: wide tables (100+ cols), composite expressions,
additional FOR clauses, deleted records during
AdsCreateIndex61. - CI now builds and runs
dbf_cdx_benchas a multi-tag smoke on every job. - Enhanced
dbf_cdx_benchto time realistic 4-tagINDEX ONsequences.
These changes attack the remaining per-row expression cost and multi-tag scan cost after the initial read-ahead fix.
INDEX ON Val(charfield)now correctly creates 8-byte FoxNumeric CDX keys (viaevaluate_index_expr_numberprobe on first record +"VAL("heuristicklen=8+FoxNumericencoding).
mark_cdx_key_encodingon reopen now promotes any CDX with keylen==8.- Robust
AdsSeekconversion for cases where rddads passes the numeric as string digits. UPPER()(both general and fast bare-field paths) now dispatches to the OEM upper table (oem_upper+ NTXPL852/PL852) when the connection uses national OEM collation. This makesUpper(field)indexes + seeks match Harbour behaviour under PL852/OEM.- Unit test coverage for the exact repro path (
Val()seek + key_length==8 on real ASORTYM CDX from the issue). - Verified end-to-end on the anonymised ASORTYM repro (14k rows, 125 cols,
16-tag bag, NTXPL852): both
Val()andUpper()seeks now OK, raw keys continue to work.
Significant speedup for local DBFCDX index creation when using Harbour
contrib/rddads (ADSCDX) or direct AdsCreateIndex* calls.
Root cause: Key collection for the bulk B+tree builder did
goto_record(r) for every record. This unconditionally called
invalidate_read_cache() on the CdxDriver and performed active index
cursor repositioning, defeating the 64 KB read-ahead block cache.
Fix:
- New internal helper
Table::load_record_for_bulk_scan(). - Changed the full-table scan loops in:
AdsCreateIndex61(main path + sibling-tag resync for multi-tag bags)- legacy
AdsCreateIndex Table::reindex()
- Now use direct
driver()->read_record_raw(r)followed by bulk buffer install. Sequential scans benefit from large reads + memcpy.
Also updated known-issues.md.
This directly addresses the ~11× slowdown vs SAP ACE 11.10 on real
production tables (e.g. ASORTYM.DBF 14k rows / 125 cols / 16 tags) and
full reindex workloads.
Reported-by: rkedzioralmaalpinex
The v1.8.0 NTXPL852 unit tests used adjacent string literals and greedy
\x escapes for the Polish Ł (OEM 0x9D) rows. Clang on Linux/macOS
treated them as -Werror,-Wstring-concatenation / hex escape sequence out of range, breaking ci and release legs for POSIX platforms
while Windows MSVC passed.
Fix: shared tests/fixtures/polish_oem_fixture.h with explicit
constexpr byte arrays; release.yml skips artifact upload when a build
leg produced no ASSET.
No functional engine changes vs v1.8.1 — use v1.8.2 release archives for Linux/macOS binaries.
Harbour rddads passes scope strings at trimmed length (hb_itemGetCLen),
but CDX index keys are space-padded to key_length. An unpadded scope on
a wide character field (e.g. work-order C(10) with setScopeTop /
setScopeBottom on the same cWrkord) made key <= bottom fail for every
matching row — GotoTop landed at EOF on both local and remote.
Fix: pad string/RAW scope keys to the active index key_length in
AdsSetScope, matching relation_child_key() and native ACE behaviour.
Test: QA-D: character ordScope honours unpadded scope on wide key in
abi_qa_repro_test.cpp.
Reported-by: production work-order labour-items filter.
Polish CP-852 national collation for CDX index build, soft/hard seek,
insert, and reindex. AdsSetCollation now accepts NTXPL852 and
PL852 (case-insensitive aliases) in addition to BINARY / NOCASE.
Ł (0x9D) sorts between L and M, matching Harbour / Clipper / SAP ACE
local-server behaviour on Central-European datasets.
- New engine module
oem_collation(256-byte PL852 sort table). CdxIndex::compare_keys_()honours the connection's OEM sort weights.apply_cdx_oem_collation()stamps collation on create/open/reindex.
Table::reindex() for CDX tags now uses clear_data() +
build_bulk() (same fast bottom-up path as CREATE INDEX) instead of
erase-then-per-record insert.
19 new unit tests across engine, CDX driver, and ABI layers
(oem_collation_test, cdx_build_bulk_collation_test,
abi_ntxpl852_seek_test, abi_ntxpl852_collation_abi_test).
OrdScope / AdsSetScope top/bottom key-range bounds were sent to the
server (wire opcode SetScope 0x98) but GotoTop/Skip ignored them:
navigation walked every record in the table instead of the scoped group.
Root cause: scope is stored on the ABI table's active order, while
GotoTop/Skip only routed through that handle when ordered_tables_
was set — which happened on SetOrder but never on SetScope. Harbour
OrdSetFocus + OrdScope flows often set scope without a preceding
SetOrder wire op (client already tracks active_index_id from
auto-opened production CDX).
Fix:
- Server
SetScopehandler: mark the parent table inordered_tables_after a successfulAdsSetScope. - Client
remote_activate_index: always syncSetOrderto the server before index-driven navigation.
Files: session.cpp, remote_index_nav.cpp.
Tests: remote AdsSetScope constrains GotoTop/Skip walk in
abi_remote_index_nav_test.cpp; openads_remote_scope_probe harness.
Two critical blockers for Harbour rddads + FiveWin TDataBase /
xBrowse over remote connections:
-
OrdKeyCount() returns 0 on remote aliases (#128):
xBrowsegrids showed no rows becauseOrdKeyCount()internally calledAdsGetKeyCount(hOrdCurrent)which had no remote code path. Added new wire opcodeGetKeyCount/GetKeyCountAck(0xB0/0xB1),RemoteConnection::key_count()client method,remote_index_key_count()bridge, and a server handler that routes through the ABI'sAdsGetKeyCountto compute the true filtered key count from the active index order (e.g. 100 for aFOR CODIGO>100tag on a 200-row table). Also added aget_remote_table()guard so table handles route correctly too. -
AdsGetDate() crashes on remote Date-type fields (#128): rddads resolves
FieldGetonADS_DATEcolumns toAdsGetDate(hOrdCurrent, ...), passing theRemoteIndexhandle.AdsGetDatedelegated toAdsGetFieldwhich only checksget_remote_table()-- never sees theRemoteIndexand falls through to a null local pointer -> ACCESS_VIOLATION. Fixed by resolvingRemoteIndex-> parentRemoteTableviahandle_for_remote_tablebefore callingAdsGetField.
Files: wire.h, client.h, client.cpp, remote_index_nav.h,
remote_index_nav.cpp, session.cpp, ace_exports.cpp.
Tests: 2 new unit tests in abi_remote_index_nav_test.cpp.
All notable changes to OpenADS are recorded here. The project follows Semantic Versioning once 1.0 ships; until then 0.x.y releases may break the C ABI between minor versions to track the real ACE SDK.
Fixes reported while using Harbour ADSCDX + FiveWin TDataBase over
remote connections (tcp:// against openads_serverd):
-
Date fields (#4):
FieldGet(andAdsGetJulian/AdsGetField) onADS_DATEcolumns (e.g.WRKDAT) no longer ACCESS_VIOLATION crashes. All remote value paths now resolve fields viaremote_field_indexfirst (handles the ordinal-as-small-pointer idiom used by X#/FWH/rddads safely) before anyto_internalor name lookup. Server-side ABI connections now forceYYYYMMDDdate format so the row cache always carries canonical 8-digit strings (matching local engine behaviour).AdsGetLong/AdsGetDoublefallbacks and memo paths also hardened. -
FieldPut on unlocked record (#6): No longer crashes with AV (or succeeds silently).
SetFieldon the server now obtains the ABI handle (viatbls_h_orensure_abi_handle) and returnsAE_RECORD_NOT_LOCKED(5035) for writes to existing records that are not locked. The classic Clipper "write value back to test lock, catch EG_UNLOCKED" idiom now works over REMOTE.LockRecord/LockTable(and unlock) are forwarded to ABI handles for regular remote tables. Engine table is still used for the actual mutation to preserveappend_record+ immediate set state machine.pending_appendis explicitly set after serverAppendBlank; post-append sets are allowed without prior lock (standard behaviour). -
Ordinal safety & repeated FieldGet (#5, #1, #2): Removed direct
to_internal(pucField)/reinterpret_castonpucFieldin every remoteAdsGet*andAdsSet*(String, StringW, Double, Logical, Julian, Long, MemoLength, FieldRaw, FileTo/FromField, etc.). All now go throughremote_field_index(which already handled the <0x10000 ordinal case). This prevents AVs on first Get after open, at EOF, on Date fields, and on the redundant second Get that some FWH helpers perform. -
Other remote robustness (#3, #7):
AdsGetAllLocksno longer returns 5000 / crashes on remote table handles (returns count=0 as safe stub; full wire impl still pending).DbInfo(DBI_FULLPATH)paths remain stable (remote name returned). Row cache / blank handling at open/EOF/BOF made more consistent so fewer workarounds are required. -
Server lock & append hygiene: Lock opcodes now also act on
tbls_h_handles.AppendBlankhandler explicitly markspending_append.SetFieldprefers the engineTable*for data writes (correct cursor after append) while using ABI only for the lock pre-check.
All changes are covered by existing remote wire unit tests (including
AdsAppendRecord + multi-Set including Date columns + navigation).
openace64.dll and openads_serverd must both be updated for full effect.
Reported by users integrating OpenADS REMOTE mode with FiveWin
TDataBase.
Harbour ADSCDX + FWH xBrowse / TDataBase with OrdSetFocus and
production CDX tags (e.g. customer.dbf / CUSTNAME on openads_serverd)
had chaotic browse behaviour: scrollbar misaligned, rows shifting on
Refresh, and the first row repeating when scrolling up from the top.
-
Bug fix:
AdsGetKeyNumreturned physicalRecNoinstead of logical key position — FWHxBrowse:SetRDD()usesAdsKeyNo(, , 1)for the vertical scrollbar. Remote indexed tables now trackcurrent_keynoacrossAdsGotoTop/AdsSkip/AdsGotoRecordand implementAdsGetRelKeyPos/AdsSetRelKeyPosagainst the active order. -
Bug fix:
GotoRecordleft the server ABI index cursor stale —xBrowse:Paint()savesRecNo(), walks visible rows viahOrdCurrent(AdsSkipon the index handle), then restores withDbGoto(bookmark). The server engine moved to the bookmark recno but the parallel ABI handle (used for orderedSkip) did not, so the next index skip walked from the paint position instead of the selected row.Session::GotoRecordnow callsAdsGotoRecord(hord, recno)when the table is inordered_tables_. -
Bug fix:
AdsAtBOFalways answered “not BOF” whilerow_valid— AfterSkip(-1)at key #1, Harbour'shb_adsUpdateAreaFlagsnever saw BOF, so FWHGoUprubber-banded and repainted the first row. The client now setsnav_at_bof/nav_at_eofwhen a skip does not change recno. -
Client:
remote_index_skip(0)—Skip(0)settles prefetch lag without clearingprefetch_consumedfirst; index-nav preamble no longer invalidatesrow_validbefore every skip ack. -
Regression tests —
remote bookmark restore keeps AdsGetKeyNum coherent,remote index skip after GotoRecord bookmark restore,remote index skip(-1) at top sets BOF for xBrowse GoUp, plus extendedabi_remote_index_nav_test/abi_remote_prefetch_testcoverage.
Reported while validating FWH testads.prg / TDataBase + xBrowse
against tcp://192.168.18.184:16262//tmp/openads_mac (Harbour rddads
unchanged).
-
Bug fix: SAP production CDX tag names corrupted —
CdxIndex::list_tags()andopen_named()decoded struct-tag compact keys incorrectly for ADS-SAP / BCC compound CDX files (e.g.customer.cdxon the iMac test dataset). Tag 2 surfaced asAMEinstead ofCUSTNAME; tag 1 asCUSTNOinstead ofCUSTNO. OpenADS now reads the canonical tag name from each sub-tag CDXTAGHEADER (+24) and forcesdup=0on the first struct-leaf key (Harbourhb_cdxPageLeafDecodeparity). FixesAdsGetIndexName/ rddadsOrdName()/ FWHTDataBase:IndexName()over REMOTE mode. -
Bug fix:
AdsGetFieldat BOF/EOF returned 5000 — FWHTDataBase:td_blankrow()doesDBGOBOTTOM+DBSKIP(1)thenFieldGeton the append-row position. OpenADS now returnsAE_SUCCESSwith a type-default blank field (spaces /Ffor logical) instead ofAE_INTERNAL_ERROR, for local tables and remoteGetFieldwire ops. Unblocks HarbourADSCDX/ xBrowse on OpenADS without patching FiveWin. -
Regression tests —
abi_no_current_record_test,engine_navigation_empty_test,abi_cdx_tag_order_test(SAPcustomer.cdxfixture),abi_remote_prodcdx_test(expectsCUSTNO+CUSTNAMEwhenOPENADS_TEST_REMOTEis set).
-
Bug fix:
ensure_abi_handle()used basename-only table paths — Whenopenads_serverdhandled remoteOpenIndexfor a table opened asorders/workorders.dbf, the lazy ABI handle was reopened asworkorders.dbfat the data root. Production CDX auto-bind then failed (AdsGetNumIndexes/ rddadsOrdCount()returned 0) even though the CDX sat beside the DBF in a subdirectory. The session now stores the originalOpenTablepayload and reopens with that relative path; theOpenTableAckproduction-bag hint is also sent relative to the data root. -
Regression tests —
remote production CDX auto-open in subdirectory(embedded server) andREMOTE: workorders subdirectory auto-binds production CDX(live server, gated onOPENADS_TEST_REMOTE).
Reported by FWH users opening large DBF/CDX tables via Harbour ADSRDD /
TDataBase over tcp:// against openads_serverd.
- Windows x64/x86 archives ship both engine DLL names at the ZIP
root:
openace64.dll/openace32.dll(OpenADS build product) andace64.dll/ace32.dll(Harbourrddads/ FWH drop-in), plus matching.libimport libraries. The release workflow now verifies all four files before publishing.
-
Bug fix:
AdsGetNumIndexesin REMOTE mode — Previously returned 0 because it queried the server engine handle (which hadn't opened the production index). Now countsRemoteTable::index_handles.size()locally, matching the production CDX auto-open + explicit AdsOpenIndex index count. This unblocks rddads'DbSetOrder(n)andOrdBagName()in REMOTE mode. -
Bug fix: implicit GoTop after
AdsOpenTable90in REMOTE mode — After table open + production CDX auto-open, the client had no record buffer, causing crashes onAdsGetField/AdsGetRecordCountetc. LOCAL mode leaves the cursor at BOF with a valid buffer; REMOTE left the buffer empty. Now sends an implicitGotoTopon table open to populate the record cache, matching LOCAL semantics. This eliminates the need for an explicitDbGoTop()afterUSEin FWH. -
New test:
abi_remote_prodcdx_test.cpp— 8 test cases that validate the complete FWH rddads workflow over TCP against a pre-existing production database (DBF + CDX): OrdBagName, AdsGetIndexName, GoTop + FieldGet by ordinal and by name, AdsSetIndexOrderByHandle, AdsSetIndexOrder by tag name, ordered full-scan, multi-table simultaneous open, and FieldGet on multiple fields after Skip. -
FieldGet by ordinal idiom documented — ACE's
ADSFIELD(n)casts a 1-based ordinal to a pointer ((UNSIGNED8*)(uintptr_t)n), NOT a string"1". Bothremote_field_index()andresolve_field_index()detect small pointer values (< 0x10000) as ordinals. Tests updated to use the correct idiom. -
Gated on
OPENADS_TEST_REMOTE— set to a server URI (e.g.tcp://192.168.18.184:16262/) to run against a live remote server. Skipped in default CI. -
DOING.md added — live working-notes file tracking in-progress investigation, test results, and pending fixes.
- Windows x86 (32-bit) archive restored — v1.5.1 shipped only
openads-*-windows-x64.zipbecause the x86 MSVC leg failed at build time (duplicateENTRYPOINTdeclarations inhttp_server.cpp/mgprobe, fixed on main). The release workflow now verifies bothwindows-x64andwindows-x86ZIPs before publishing; the x86 ZIP bundles prebuiltlib/msvc/ace32.lib(stdcall) plusopenads_ace_x86.deffor Harbourrddads.
- Harbour smoke — green on GitHub Actions (
contrib/rddadsbootstrap, freshopenace64.liblink).
- Path jail on remote Connect — client paths are canonicalized and
confined under
openads_serverd --data; traversal attempts are rejected. - LockMgr nested unlock — OS byte locks remain held until the final
nested
unlock_*releases them. - Remote field writes —
AdsGetMemoDataType,AdsSetStringW,AdsSetJulian, andAdsSetFieldRawroute throughtcp://. - TLS — peer certificate verification on by default;
OPENADS_TLS_INSECURE=1for dev/self-signed endpoints.
harbour-smokejob in GitHub Actions (Windows).tools/scripts/run_harbour_smoke.ps1andbootstrap_harbour_ci.ps1— portable Harbour bootstrap for CI.
AdsSetRelation/AdsSetScopedRelation— parent→child relations on local andtcp://tables;apply_relations_for_handle()after navigation.AdsSetRecord/AdsGetRecord— wire opcodes0xA8–0xAB.AdsCustomizeAOF— wire opcodes0xAC/0xAD.AdsAggregate/AdsFetchWhere— local in-process DBF tables (SQL backends via existing aggregate path).
- SQLite write —
AdsAppendRecord,AdsSetString,AdsWriteRecord,AdsDeleteRecordwith rowid-keyed DML and parameterized binds (abi_plus_sqlite_write_test.cpp, in-process). - MSSQL native (TDS) write — same ABI surface; PK discovery via
INFORMATION_SCHEMA, staging buffer,SELECT *refetch after DML (abi_plus_mssql_write_test.cpp, gated onOPENADS_TEST_MSSQL_CONNSTR). - MSSQL read fix —
ADS_STRINGfields padded to declared width inmssql_get_field(NVARCHAR live read test).
- ADT/ADI fixtures in
tests/fixtures/adi/+ generatorgenerate_adi_fixtures; smoke tests no longer skip for missing files. - VFP header
0x32— autoinc and nullable columns together;_NullFlagssynthetic column when the NULL bitmap is present.
BackendTxManagerhooks —AdsBeginTransaction/ commit / rollback /AdsSetAutoCommiton SQLite and PostgreSQL; DML auto-commit after write.- Tier-1 utilities in execution — field optimizer and where builder drive actual SQL generation on SQLite reads.
- AOF V2 —
Like/IsNullops handled inaof_eval(clang-Wswitch). - Harbour CI bootstrap — track
harbour/coremaster. - Concurrent SQLite test — tolerate minor
SQLITE_BUSYunder contention.
BackendTxManager: nested transactions + auto-commit. Shared transaction manager embedded in every SQL backend connection. Supports nested BEGIN/COMMIT with SAVEPOINT emulation, auto-commit after N DML statements (configurable via connection string), and dirty-flag tracking. SQLRDD reference:SR_CONNECTION:nTransacCount,nAutoCommit,nIteractions.BackendFieldOptimizer: lazy column loading with learning. Tracks which columns are actually read per table. AfterLEARNING_THRESHOLD(5) unique single-column fetches, switches toSELECT *to avoid repeated demand-fetches. Integrated intoSqliteTableandPostgresTable. SQLRDD reference:SR_WORKAREA:sqlGetValue,FIELD_LIST_*.BackendWhereBuilder: restrictor composition. Combines For clause, user filter, scope bounds, index restrictions, AOF predicates, and recno filters into a single AND-ed WHERE clause. Handles exact seek (lower == upper collapses to=) and range seek. SQLRDD reference:SR_WORKAREA:SolveRestrictors.BackendTableOpsvtable: transaction ops. Newbegin_tx,commit_tx,rollback_tx,set_auto_commitfunction pointers in the backend vtable. SQLite and PostgreSQL adapters registered.
- 50+ new translatable functions. The
try_emit_sql_where()emitter now handles STR, VAL, DTOS, DTOC, CTOD, ROUND, CEILING, CEIL, MOD, EXP, LOG, LOG10, SQRT, SIGN, PADR, PADL, PADC, STRTRAN, LEFT, RIGHT, AT, ATNUM, DATEADD, DATEDIFF, IIF, IF, NIL, ISNULL, ISBLANK, EMPTY, LEN, YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, DOW, CDOW, CMONTH, NOW, and more. Unsupported functions (RECNO, DELETED, REPLICATE, SPACE, STUFF, OCCURS) decline cleanly. $contains: field-to-field support.field1 $ field2now emitsfield2 LIKE '%' || field1 || '%'(or CONCAT variant). Literals with LIKE wildcards (% _ ) still decline to avoid semantic mismatch.SqlDialectexpansion. New fields:length_fn(LEN → LENGTH/CHAR_LENGTH),now_fn(DATE() → NOW()/CURRENT_DATE),true_literal/false_literalfor .T./.F. rendering.
UNION [ALL]SELECT support. The SQL parser now handlesSELECT ... UNION [ALL] SELECT ...with any nesting depth. Parsed viaSelectStmt::UnionMemberlist; each member carries its own FROM, WHERE, ORDER BY, LIMIT, and aliases. Full round-trip through ADS query execution.
- DDL statement parsing. New
AlterTableStmt,DropTableStmt,DropIndexStmtstructs with full parser support. Identifiers, quoted names, and IF EXISTS clauses are all handled. Ready for backend execution hooks.
- LIKE operator.
NAME LIKE 'A%'now parses and round-trips in the AOF expression layer with full%and_wildcard support. - IS NULL / IS NOT NULL. Unary null-test operators added to the AOF filter expression grammar.
- N-way comma join (3+ tables).
FROM a, b, c, d, enow parses and executes with an arbitrary number of tables (was limited to exactly 2). Left-deep execution plan with hash-join on composite keys. Filter pushdown pushes WHERE residuals to the deepest join level. Pinned bysql_parser_testandabi_cdx_conditional_index_test. <alias>.*wildcard projection.SELECT line.*expands to all columns of the aliased table, matching ADS behaviour.UPPER(col)scalar function in WHERE. Parsed and mapped to a case-insensitive comparison, soWHERE UPPER(name) = 'SMITH'now works end-to-end.FROM t AS atable alias on the base table. Previously only consumed for derived tables; now accepted on plain table names.- Brackets
[file.dat]for free-table names in FROM. Theread_identifier_or_filename()parser now handles[...]syntax, matching ADS canonical free-table references. WHERE 1 = 1constant folding. Always-true predicates are folded at parse time, eliminating unnecessary runtime evaluation.- ODBC temporal literals.
{d 'YYYY-MM-DD'},{ts ...},{t ...}are now parsed and accepted in SQL.
- Bulk-load index builder (
build_bulk). New bottom-up B+tree construction path forCREATE INDEX— approximately 10× faster than record-by-recordinsert()on large tables. The builder sorts keys in-memory and emits a complete B+tree in a single pass. - O(1) browse position cache.
ordered_recnos_cached()andpos_of_recno_cached()onCdxIndexcache the key↔recno mapping soAdsGetRelKeyPos/AdsGetKeyNumanswer from an in-memory vector instead of walking the index. - CDX conditional (FOR) index predicates — persist + apply.
CREATE INDEX ... FOR <condition>now persists the condition in the CDX sub-tag header and applies it at insert time — only records satisfying the FOR clause get indexed. Full round-trip through reopen. Pinned byabi_cdx_conditional_index_test. - CDX flush-skip for read-only. Opening and closing a CDX file no longer triggers a flush when no page is dirty.
- CDX FOR-clause hardening. Fails loud instead of silently dropping or truncating unparseable FOR clauses.
- NTX empty-but-rooted leaf on PACK/reindex. Fixes error 5004
when reindexing an NTX that had empty leaves left by prior
erase()calls. Pinned byabi_ntx_pack_reindex_test. - Composite CDX key width not pinned to the 254-byte probe. Follow-up to the v1.2.3 character-key fix (PR #68): a composite key expression no longer derives its on-disk width from the 254-byte evaluation probe — it uses the actual key width, so composite tags stay the right size and interoperate with native readers.
- Server-side filtered scan (
FetchWhere). NewFetchWhereopcode (0xA4) lets the client send a Clipper-style FOR predicate and receive only matching rows — reducing round-trips and bandwidth for non-AOF predicates. Evaluated with the same engine evaluator used for CDX FOR index conditions. Documented indocs/wire-protocol.md§5.22.
- Sharded-reactor connection pool (
WorkerPool). NewWorkerPoolclass multiplexes many client connections over a fixed pool of worker threads (default OFF viaOPENADS_SERVER_POOL=ON). IncludesFrameReaderfor non-blocking partial-frame buffering andSessionclass extracted fromserver.cpp. Stress harnesses:tools/stress/remote_random_main.cppandtools/stress/remote_concurrency_main.cpp. EnterpriseConfigsingleton. Environment-driven tunables:OPENADS_SERVER_POOL(enable pool),OPENADS_SERVER_POOL_WORKERS(thread count),OPENADS_SERVER_MAX_SESSIONS(connection cap), pool toggles for ODBC/SQLite/OLEDB backends.- Session reaping + max-sessions cap. Abandoned connections are
reaped after a timeout; a hard cap prevents thread exhaustion
under load. Deadlock-free
stop()lifecycle.
- PostgreSQL column metadata via information schema.
AdsDDGetFieldPropertyfor PostgreSQL tables now exposesIS_NULLABLEandCOLUMN_DEFAULTviainformation_schema.columns. - SQL concurrency safety — stmt_map serialisation. Concurrent SQL statement execution no longer corrupts the internal statement map; access is serialised.
- SQLite busy-timeout + WAL mode. Contended SQLite writes no
longer fail with
SQLITE_BUSY; a busy-timeout and WAL journal mode are enabled at connection time. - SQL CREATE TABLE honours statement table type.
CREATE TABLEandCREATE TABLE ... ASnow respect the type specified in the statement (e.g.ADS_ADT).
- ADT companion stream count.
AdsCreateTable(ADS_ADT)now writes the correct ADT header companion-type count instead of a flat 1.
- Connection / handle introspection.
AdsGetConnectionTypereportsADS_REMOTE_SERVERfor a remote handle (local otherwise);AdsGetHandleTypedispatches on the registry handle kind (connection / table across all backends / statement);AdsGetIndexCondition/AdsGetIndexFilenamereturn real values instead of empty stubs.
- Strict-warning (
-Werror) cleanups indata_dict.cpp. Explicit casts inle16()and the\uXXXXescape loop, and removal of two dead static helpers (trim,split_tabs), so the data dictionary compiles clean under clang/gcc-Wconversion/-Wsign-conversionand MSVC/WX(C4505).
- xBase++ smoke test. New
tests/xpp/directory with a raw-ACE smoke test viaDllPrepareCall, plus translations (ES, PT). Runner:tests/xpp/run.sh. - FiveWin ORM cookbook. New
cookbook/orm/fivewin/with agrid_orm.prgexample, FiveWin build script, and README. - CDX empty-table key-width edge test. Verifies correct key width for composite expressions on an empty table.
- Concurrent SQL + SQLite contention tests.
abi_sql_stmt_concurrency_testandsqlite_concurrency_testvalidate thread-safety under contention.
- CDX index direction fix for Harbour rddads (FiveWin).
AdsCreateIndex61decodeddescending = ulOptions & ADS_DESCENDING (0x08). Instrumenting the two RDD clients showed they put the compound/descending option bits on swapped positions: X#'s ADSRDD sends0x02for an ascending tag and0x0Afor descending, while Harbour'srddadssends0x08for ascending and0x0Afor descending. So a plain HarbourINDEX ON f TAG t(0x08) was read as descending and every Harbour/FiveWin index was built reversed —AdsGotoToplanded on the last key andSkipwalked backward, so aTBrowse/tDatabasegrid showed its rows upside-down (Seekstill worked, which masked it). Direction is now decoded as descending only when both0x02and0x08are set (0x0A); a lone0x02or0x08is that client's compound marker and is ascending. The SQLCREATE INDEXpath emits0x0Afor a descending tag so it round-trips through the same decode. Pinned byabi_cdx_index_direction_testandexamples/fivewin/tdata_index_test.prg. - Build fix: drop a dead
trim()indata_dict.cpp. An unreferenced static function tripped-WXC4505 on a clean MSVC build.
- CDX character index key width fix (PR #68).
AdsCreateIndex61derived a character tag's fixed key width from the trimmed value of the first record. When the first row was short (e.g."ANA") and later rows shared a longer prefix ("ANABELA CARDOSO","ANABELA FERREIRA"), every later key was truncated to the first row's width and collapsed onto the same stored key, so distinct values became indistinguishable and a seek landed on the wrong record — both inside the index and for native FoxPro/Clipper readers of the bag. The key width now comes from the declared field length for a bare character field, falling back to the untrimmed first-record width for a composite expression, keeping the 32-char default only for an empty table. Numeric CDX/NTX key widths are unchanged. Pinned byabi_cdx_char_keylen_test. - Build fix:
<cstdint>insqlite_uri_test.std::uint8_twas used without including<cstdint>; clang/libc++ does not pull it in transitively, so theninja-clang-WerrorCI job failed while MSVC and AppleClang stayed green. Added the explicit include. - Full unit suite 739/739, 0 regression (was 738).
- CDX empty-leaf walk fix (PR #63). Forward and backward
index walks now skip empty leaves left behind by
erase(). Previously,seek_first/seek_key/nextstopped at the first empty hole andprevfollowed the left-sibling pointer into it, reporting end-of-index while live keys remained in later (or earlier) leaves. A sharedskip_empty_leaves_right_/skip_empty_leaves_left_helper advances over holes. Fixes REINDEX / bulk-deleteADSCDX/5000(record number out of range). - CDX leaf recno bits + prefix seek (PR #62). Two correctness
fixes: (1)
compute_layoutnow sizes the record-number field frommax_rec(not just key length), so tags with wide keys (≥40 bytes) no longer silently truncate recnos ≥ 4096; (2)seek_keycompares only the search-key length, so a partial (prefix) seek likeSEEK "ART-00024800"matches a stored"ART-00024800 desc ..."key. A guard refuses encode whenmax_rec > rec_mask, failing loudly at write time. - MSSQL backward SKIP off-by-one (PR #65).
MssqlTable::skipusedabs_n >= posfor the backward branch, so a SKIP landing exactly on row 0 reported BOF. Changed to>soabs_n == posreaches index 0 (a valid row). - ABI typed getters + AdsGetIndexHandle for SQL backends
(PR #66).
AdsGetDouble/AdsGetLong/AdsGetLongLong/AdsGetStringnow dispatch through the per-backend ops vtable, so PostgreSQL (and other SQL backends) return real values instead of error 5000.AdsGetIndexHandleresolves by-name for PG tables so indexed seek works end-to-end. - NTX numeric key edge-case tests.
ntx_numeric_key()pure function now tested for -0.0 normalisation, width/dec clamping, negative byte-complement, and large-value truncation. Custom-key add/delete on numeric NTX index covered. - CDX empty-tree + prefix-seek edge tests. Empty tree (seek_first/seek_last/seek_key return AfterEnd), all-erased tree (forward walk crosses empty leaves), exact-length prefix match, and descending prefix seek.
- Full unit suite 738/738, 0 regression (was 726).
- NTX numeric key format fix (PR #67). Numeric fields indexed
into an NTX bag now store keys in the native DBFNTX form
(zero-padded magnitude + complemented negatives) instead of
space-padded
STR()text at a probed width. A native xBase reader'sdbSeek(<number>)now matches the on-disk key for positive, decimal, and negative values. Reopened index bags retain the numeric encoding.abi_ntx_numeric_key_testasserts the native byte layout; full unit suite 720/720, 0 regression. - Added unit tests: adm_memo, codepage, maria_uri, postgres_uri, proc, sqlite_uri (710 new lines, 706/706 tests pass).
- Remote benchmark docs: iMac WiFi (784K rec/s) and charleskwon.com SSH tunnel (676K rec/s) with 500K-record results.
- Removed IMAC_CONNECTION.md from tracking (contains credentials).
- ORM examples synced to v1.1.0-alpha.
-
Deferred-flush bulk-insert mode (528× speedup). A new
AdsSetDeferredFlush(hTable, 1)API puts the table into deferred-flush mode:AdsWriteRecordwrites the record to OS cache but skips the per-recordFlushFileBufferscall. Data is flushed to physical media only whenAdsFlushFileBuffersis called explicitly (or on table close). 500K records + CDX index build completes in ~26 seconds (19s bulk insert at 26,381 rec/s + 7.2s CDX build + 36ms final flush) vs. ~2.7 hours before (50 rec/s). Remote benchmark (Windows client → iMac server over WiFi tcp://): 500K records in 0.69s at 784K rec/s — 36× faster than local mode. 649/649 unit tests pass; backward-compatible — default behaviour is unchanged (flush on every write). -
MSSQL native TDS 7.4 backend (PR #53 integration). Native SQL Server connectivity via the TDS 7.4 wire protocol with optional mbedTLS encryption. Supports connect, authentication (SQL/Windows), table open, field read, and navigation. URI scheme:
mssql://user:pass@host:port/database. Enabled viaOPENADS_WITH_MSSQL=ONCMake option (requiresOPENADS_WITH_TLS=ON). 649/649 unit tests pass.
- SQL backends: PostgreSQL / MariaDB / ODBC behind a pluggable
backend-ops registry (PR #31). OpenADS can now open tables on
PostgreSQL, MariaDB / MySQL and any ODBC-reachable engine behind
the ACE ABI, selected by the connection URI (
postgresql:///mariadb:///odbc://) exactly like the SQLite backend. Navigation, field read and column SEEK work; write is per-backend. The four SQL backends register oneBackendTableOpsstruct each (17 function pointers), so the ~17 ABI navigation / field functions stay backend-agnostic instead of multiplying a per-backendifblock — adding a further backend is one ops struct plus one registration line. Identifiers are validated to safe ASCII and SEEK values use prepared-statement parameters. The native local DBF / ADT andtcp://remote paths are unchanged fall-throughs. Seedocs/OPENADS_PLUS.md. Verified: full unit suite 572/572; PostgreSQL/MariaDB/ODBC e2e (41/45/59 assertions) against live servers on the contributor's side.
- CDX stale record-count refresh on the fetch path (PR #50). A
CdxDrivercaches the DBF record count atopen(). In a multiuser deployment a peer connection can append rows afterward, leaving that cache lagging; an index walk that reached a just-appended recno (e.g. mid-REPLACE … FOR/ DBEVAL) then failed hard with a spurious ADSCDX error 5000.read_record_raw/write_record_rawnow re-read the on-disk count under a shared header lock before declaring a recno out of range, with an unlocked-refresh fallback. Slow path only — a normal forward scan never reads past the count, so the single-writer case pays nothing.
- Round-trip-thrifty remote scan (PR #47). A forward scan over
the
tcp://wire no longer costs ~one TCP round-trip per record. A sequential-prefetch path — negotiated via a Connect capability flag — piggybacks a lookahead block onto forward-Skipacks; the client serves them locally and folds the consumed count back into the next wire step so the server cursor never desyncs.AdsAtEOF/AdsAtBOFare answered from the cached current row andAdsIsFoundfrom a cachedFound()flag. A 50k-record loopback scan is ~3.9× faster (NAV-only) / ~3.3× (3-field read),IsFoundround-trips drop to zero. Additive and backward-compatible: clients that don't advertise the capability keep the previous wire behaviour. - Cookbook expansion (PR #46). New
console/examples (SQL viaAdsExecuteSQLDirect, native ADT withADSADT+.adi, atcp://remote client), a FiveWinxbrowseCRUD sample, and an all-back-ends ORM benchmark (orm/complete/) with a cross-back-end content checksum and a seek-vs-scan headline.
- Responsive Studio web console. The Studio SPA
(
tools/serverd/spa_index.h) now adapts to phones and tablets: the table-list sidebar collapses into a slide-in drawer (☰ in the header, dimmed backdrop, auto-close on select) below ~768 px; tabs scroll horizontally; on phones forms stack to one column, modals fit the viewport width, and touch targets are enlarged. Also fixes a pre-existing dark-theme bug where--panel/--panel-2/--borderwere self-referential CSS variables, so panels and borders rendered transparent.
SKIPhonoursSET DELETED ONin natural order.Table::skipon an unindexed table stepped straight onto deleted rows; it now skips deleted records (matching the index-order path andGOTO TOP/GOTO BOTTOM) whenSET DELETEDis ON. Fixesabi_deleted_records_test("middle records deleted: Skip sees only live rows"), which had been failing the test step on every CI platform.- Native ADT / ADI create, read, write, and index seek (PR #41).
OpenADS now operates end-to-end on native
.adt/.adi/.admfiles:AdsCreateTable(ADS_ADT)writes a valid header + field descriptors (+ optional.admmemo store),AdsAppendRecord/AdsWriteRecordpersist rows and memo payloads, re-open + field get- memo round-trip on read,
AdsCreateIndex61builds.adibags, andAdsSeekworks on character and numeric ADI keys. AUTOINC counter is seeded from existing rows at open.
- memo round-trip on read,
- POSIX platform hardening.
file_posixstores handles as(fd+1)so a real fd 0 (stdin closed) is not mistaken for the not-open sentinel;pread/pwriteretry onEINTR;map_readonlyrejects zero-length maps;LockMgrrefcounts repeated locks and releases the OS lock only on the final unlock;TxLog::read_allbounds-checks every UPDATE / APPEND field length against truncated / corrupt WAL. - macOS / clang build fixes. Resolved
-Werrorbreaks introduced by the ADT/ADI work: sign-conversion inadi_index.cppand theenviron/ dangling-pointer issues in the ADT scope-validation test. - Documentation. New SQLite backend guide (
sqlite://connection URI,?key=encryption, field-type mapping, limitations) and stored procedures guide (custom AEPCREATE/EXECUTE PROCEDURE+ the built-insp_*Data Dictionary procedures), all in EN / ES / PT. - Cookbook (PR #44). New
cookbook/folder with runnable, heavily-commented Harbour examples — aconsole/track (pureADSCDXxBase) and anorm/track (CRUD across SQLite / DBF / PostgreSQL / MariaDB / ODBC back-ends), plus connection-string, field-type and troubleshooting guides.
- Turnkey
hbmk2(.hbp) example for Harbour apps —examples/harbour-hbmk2/. Reported on the FiveTech forum: "alguna alma caritativa que proporcione un archivo de compilación.hbppara crear un programa con OpenADS — todos mis intentos han fracasado". The repo now ships a completehbmk2project:openads_demo.hbp(x64),openads_demo_x86.hbp(32-bit),openads_demo.prg(console app exercisingAdsConnect→DbCreate→INDEX ON UPPER(NAME)→dbSeek), Windowsbuild.cmdand POSIXbuild.shwrappers. Drop in your.prg, pointOPENADS_LIBat OpenADS' build output, runhbmk2. The.hbpis intentionally minimal — only the two link entries that change for OpenADS (-lrddadsplus-L${OPENADS_LIB} -lace64). - Docs walkthrough — en / es / pt. New "Build your own
Harbour app against OpenADS (
hbmk2/.hbp)" section indocs/{en,es,pt}/getting-started.mdand the matching README, including a troubleshooting table for the typical "unresolved external symbolAdsConnect60" / "rddads.libnot found" / "loaded the wrongace64.dll" pitfalls so a first-time user can self-diagnose without filing an issue.
- ADT / ADM support (M4 ADT). OpenADS now opens
.adttables produced by SAP Advantage and writes records back;.admmemo stores auto-attach when the table carries Memo / Binary fields. Full 13-type field vocabulary (CHAR, CICHAR, LOGICAL, DATE, DOUBLE, INTEGER, SHORTINT, MEMO, BINARY, TIME, TIMESTAMP, AUTOINC, MONEY). ADM uses 256-byte fixed blocks; the 9-byte in-record reference is resolved transparently by the engine.AdsCreateTable(ADS_ADT)still produces a DBF (ADT creation deferred); ADI index files not yet implemented; SAP proprietary ADT encryption not yet supported. Verified againstf:\pmsys\data\landlords.adtviatests/unit/abi_adt_smoke_test.cpp(skipped on machines without the fixture). - CI — macOS leg switched to a single universal (arm64 + x86_64) binary instead of separate Intel / Apple-Silicon legs.
- Harbour patch — restored the blank context line in the
rddads.hhunk oftools/harbour_patch/rddads-compat.patchsogit applysucceeds on a pristine Harbour tree.
AdsGetFieldpads CHARACTER fields to the declared width. Reported by Pritpal Bedi: a Harbourmini_xbrowse /ads(ADSCDX → OpenADS) showed every text column truncated —CharlieasCharl,BarcelonaasBarcel— while the native DBFCDX run rendered them full. Root cause:AdsGetFieldreturned CHARACTER values with trailing spaces stripped (make_stringindbf_common.cpprtrims, anddecode_fieldused it for the Character branch). DBF/xbase CHAR fields are fixed-width space-padded;FieldGetof aC(20)field must return 20 characters. With the trimmed value, xbrowse auto-sized each column to the current row's value length and clipped every other row.AdsGetFieldnow re-pads CHARACTER values to the field's declared width on the way out — both the local and the remote (wire) read paths. The engine's internal decode is left trimming, so SQL comparisons, index keys and AOF filters are untouched; verified bytests/smoke/harbour/fieldlenprobe.prg(ADSCDX now matches the DBFCDX baseline) andidxprobe.prg(index walk stillSORTED=YES, full suite 397/397).tools/harbour_patch/rddads-compat.patchapplies again. Reported by Pritpal Bedi:git applyrejected the patch withpatch failed: contrib/rddads/rddads.h:67. A prior edit had dropped the blank context line after#include "ace.h", so the hunk carried five context lines while its header still declared six. The missing line is restored; verifiedgit applyapplies cleanly to a pristine Harbourcontrib/rddadstree.- PHP binding — result-fetch and index seek.
bindings/phpgainsCursor::fetchAssoc()/fetchNum()(single-row fetch,nullpast the last row) andTable::seek()(index key seek viaAdsGetIndexHandle+AdsSeek). 37 PHPUnit tests. - CI — the release workflow gains a macOS Intel (x64) build
leg, so releases ship a
macos-x64archive alongsidemacos-arm64.
-
PHP binding —
bindings/php. Reinaldo Crespo asked whether a modern PHP extension for ACE existed: the proprietary Advantage PHP extension stopped working around PHP 5.2 and was never modernised. OpenADS now ships its own.openads/openads-php— a pure-PHP Composer package, no compiled C. It loadsace64.dll/ace32.dll/libace*.sothrough PHP'sext-ffiand wraps it in a modern namespaced OOP API:Connection,Statement,Cursor(a\Iteratorover result sets),Table,Record. Requires PHP 8.1+.- Local and remote in one path. A
Connectiontakes a local data-directory path or atcp:///tls://URI;AdsConnect60dispatches on the URI, so the binding has no mode branching. - Parameterised SQL.
Statement::query()accepts?positional or:namenamed parameters. OpenADS ACE has no host-variable binding, soParameterBindersubstitutes values client-side with per-type quoting (single-pass, so a value containing a:tokensubstring cannot corrupt the statement) — the anti-injection boundary, with its own unit tests. - Pinned by 31 PHPUnit tests (21 unit + 10 integration against
a live engine) plus a CI leg that builds the ACE library and
runs the suite. Design / plan under
docs/superpowers/{specs,plans}/2026-05-16-php-bindings*.
-
SQL
''string-escape fix.read_string_literalin the SQL parser scanned to the next'with no escape handling, so the ANSI-standard doubled-quote escape ('O''Brien') parsed as the stringOfollowed by a stray token — error 7200. Any SQL client inserting a string containing an apostrophe failed. The parser now decodes''to a single'; the unterminated-literal error path is unchanged. Pinned by a newsql_parser_testcase.
-
Index correctness sweep. Three bugs that broke CDX/NTX ordered access from Harbour
rddadsand X#'sADSRDD:AdsCreateIndex61decoded the wrong option bit.ace.hsetsADS_UNIQUE 0x01,ADS_DESCENDING 0x02,ADS_CUSTOM 0x04,ADS_COMPOUND 0x08. The descending flag was read asulOptions & 0x08— that bit isADS_COMPOUND, which bothrddadsandADSRDDset for every CDX/NTX tag. Every order built descending:AdsGotoToplanded on the last key andSKIPwalked backward. A stale comment and theabi_create_index61test had the bit values swapped the same way, so the bug was self-consistent and hidden. Now decoded with the namedace.hconstants;AdsCreateIndex90delegates to 61 and is covered too.ALIAS->FIELDqualifiers in index expressions. HarbourINDEX ON CUST->NAMEpasses the literal text"CUST->NAME"to the RDD.evaluate_index_exprcould not parse it — the tokenizer dropped-and>, so the alias parsed as an unknown identifier and every key evaluated blank, degenerating the index to record order.strip_alias_qualifiers()now removes any<ident>->qualifier (bare and nested, e.g.UPPER(CUST->NAME)) before evaluation.AE_NO_CURRENT_RECORD(5026) for not-positioned reads. Reported by Pritpal Bedi: a HarbourTBrowseover anADSCDXtable failed mid-paint withADSCDX/5000 table not positioned on a record.Table::read_fieldreturned the generic 5000 (AE_INTERNAL_ERROR) for not-positioned reads;rddadsspecial-cases 5026 as the graceful read-past-EOF path and raises every other code as a hard error.table.cppnow returns the SAP-canonical 5026.
Verified:
idxprobe.prgindex walk matches the DBFCDX baseline,posprobe.prggoes from 6ADSCDX/5000raises to 0, full suite 395/395.
-
AdsMg*server-telemetry subsystem. Reported by Pritpal Bedi after running Harbour'scontrib/rddads/tests/manage.prgagainst OpenADS: every management figure printed0— uptime, connections, work areas, comm packets, worker threads, memory. The ~17AdsMg*functions were placeholder stubs (zero-fill the caller's struct, returnAE_SUCCESS). They now report real telemetry.MgCollector(src/mgmt/) — single source of truth. Formats the SAP-canonicalADS_MGMT_*structs from aMgSnapshot. Runs identically for local-mode calls and for the server answering a remote request, so the two paths cannot diverge.MgSnapshotcarries everything across the wire: live counts (connections / work areas / tables / users / worker threads), per-entity lists, process RSS, listener port, and the cumulativeMgStatsvalues (uptime, comm packet totals, server-initiated disconnects, high-water marks).- Transport. New
MgConnect/MgRequestwire opcodes (0xA0..0xA3).AdsMgConnectto ahost:portvalidates reachability with an eagerMgConnecthandshake; a drive path such as rddads'"C:"resolves to a local-mode backend. Local mode enumerates the in-process ABI handle registry. - Honesty. Fields with no OpenADS analogue — checksum failures,
NetWare-era ECB counts, per-category memory, serial number —
report a real
0, documented indocs/superpowers/specs/2026-05-16-adsmg-telemetry-design.md. - Verified end-to-end against a live remote
openads_serverd:manage.prgnow prints real uptime, packet counts, worker threads, ports and server RSS.tests/smoke/harbour/manage_probe.prg(a non-interactivemanage.prgvariant) and the newtools/mgprobeCLI (openads_mgprobe host:port) reproduce it.
-
Harbour
contrib/rddadsclean-compile sweep. Reported by Pritpal Bedi after he hiterror: too many arguments to function 'AdsSetScope'building Harbour's unmodifiedcontrib/rddads/against OpenADS'sace.h. Compiling the whole contrib (not just the rddtst happy path) exposed a family of signatures that the 442/442 rddtst harness never reached. End-to-end repro:HB_WITH_ADS=…/openads/include/openads hbmk2 contrib/rddads/rddads.hbp -comp=mingw64now exits 0.Functions brought to SAP-canonical shape (header + ABI export + affected unit tests, plus wire opcode for the one function that round-trips over the network):
AdsSetScope— 3-arg(hIndex, usScope, pucKey)→ 5-arg(hIndex, usScope, pucScope, usLen, usDataType).SetScopewire opcode now carriesusDataType; key length derives from trailing payload size. Export mirrorsAdsSeek'sADS_DOUBLEKEY → ASCII-padded numericconversion so a scope set with adoublecompares apples-to-apples against the index's stored key bytes.AdsGetVersion— 4-arg with mistyped letter/desc slots (UNSIGNED32*x4) → 5-arg(UNSIGNED32* major, UNSIGNED32* minor, UNSIGNED8* letter, UNSIGNED8* desc, UNSIGNED16* descLen). Previous shape would have stomped 3 extra bytes past&ucLetteron the caller's stack.AdsCopyTableContents— 2-arg → 3-arg withusFilterOption. Filter mode currently accepted and documented;IGNOREFILTERSis the implemented path.AdsCreateSavepoint/AdsRollbackTransaction80— 2-arg → 3-arg with reservedulOptionsparameter (matches ACE 8.x).AdsGetAOF—(ADSHANDLE, UNSIGNED32*, UNSIGNED32*)(returned-record-count style) →(ADSHANDLE, UNSIGNED8* pucFilter, UNSIGNED16* pusLen). Returns empty filter for now; full AOF-source-string round-trip lands with M-AOF.4.AdsEvalAOF— 3rd arg wasUNSIGNED32* pulRecords; SAP expectsUNSIGNED16* pusOptLevel(returnsADS_OPTIMIZED_*).AdsSetStringW/AdsGetStringW/AdsGetFieldW— field name was declaredUNSIGNED16*(wide). SAP keeps field names ASCII (UNSIGNED8*) even on the W variants; only the data buffer is wide-char. Helperresolve_field_index_wretyped to match; UTF-16 → UTF-8 transcode dropped from the name path (it was only there to compensate for the wrong type).AdsGetString/AdsGetLong— exports were already implemented but missing from the public header, so Harbour's ANSI-path code inads1.clinked only via-Wimplicit-function-declarationwarnings. Declarations added.ADS_MAX_PARAMDEF_LEN—#defineis now#ifndef-guarded so a Harbour-style pre-define (#define ADS_MAX_PARAMDEF_LEN 2048before the include) is honoured silently.AdsGotoBookmark60— was 2-arg(hObj, *pucBookmark); SAP / real ACE is 3-arg(hObj, *pucBookmark, ulLength). Real ACE supports variable-length bookmarks (size depends on the index/order), so the caller hands the length back fromAdsGetBookmark60's*pulLengthout-param. The 2-arg form was internally inconsistent with the Get half of the same pair. Both unit tests that exercised the round-trip were updated.AdsGetAllTables— was 2-arg(*ahTable, *pusArrayLen); needsADSHANDLE hConnectas the first arg. With no connection handle the function can't know whose tables to enumerate in a multi-connection process. (Body remainsAE_FUNCTION_NOT_AVAILABLEuntil M-13 implements enumeration.)
Why rddtst missed all of this: rddtst exercises the RDD via
dbUseArea("ads")so the cursor walks Harbour's vtable, not theHB_FUN_ADSVERSION/HB_FUN_ADSCREATESAVEPOINT/ etc. PRG-level wrappers inadsfunc.c. Compiling the whole contrib is the real header check; we now have it (C:\harbour-git\ contrib\rddadsmingw64 build) and will keep it green.
- M12.25 —
AdsCreateTablestamps the DBF header last-update date. Follow-up to M12.24 (Robert van der Hulst): a freshly created+opened table reportedAdsGetLastTableUpdate()=1900-00-00until the firstDbAppendrewrote the DBF header. ACE writes the create date into header bytes 1..3 up front, so OpenADS now does too — inAdsCreateTableand every other path that lays down a fresh DBF (AdsRestructureTable/ convert, SQLINTO/SELECTresult cursors, GROUP BY / aggregate scratch tables). Newstamp_dbf_header_todayhelper uses the same UTC clock asCdxDriver::rewrite_header_, so the create stamp and the first-append stamp agree.
- M12.24 —
AdsGetLastTableUpdatereal signature + AOF non-optimisable handling. Was a 3-zero stub with the wrong signature (SIGNED32* date, SIGNED32* time); now matches ACE (UNSIGNED8* pucDate, UNSIGNED16* pusLen), reads the DBF header's last-updated stamp (header bytes 1..3, year offset from 1900) — over the wire too via a newGetLastTableUpdateopcode — and renders it through the date display format. AdsSetDateFormatstores a process-wide format string thatAdsGetDateFormat/AdsGetLastTableUpdatehonour (CCYY/YYYY/YY/MM/DDtokens; defaultyyyy-mm-dd).AdsSetAOFno longer fails error 7200 on a non-optimisable expression (e.g.Empty(NAME),UPPER(NAME) = 'A') — ACE treats those as "not optimised", so OpenADS drops any prior AOF, reportsADS_OPTIMIZED_NONE, and returns success, letting the client RDD apply the filter itself (same on the server-sideSetAOFhandler).
OPENADS_WITH_HTTPdefaults to ON. Studio no longer needs the explicit CMake flag — pass-DOPENADS_WITH_HTTP=OFFto opt out.AdsGetKeyNumreturns the correct relative key position.- FiveWin + xBrowse over ADS sample under
examples/fivewin/.
- M12.22 — versioned ACE overloads for the X# RDD. Exports the
Ads*NNentry-point names X# binds by name (AdsConnect26,AdsCreateTable71/90,AdsOpenTable90,AdsCreateIndex90,AdsDDAddTable90,AdsDDCreateRefIntegrity62,AdsFindFirstTable62/AdsFindNextTable62,AdsGetDateFormat60,AdsGetExact22,AdsReindex61,AdsRestructureTable90). Most forward to the base signature (dropping the charset / collation / page-size params newer ACE builds added);AdsGetBookmark60/AdsGotoBookmark60round-trip the recno as a 4-byte blob;AdsCancelUpdate90/AdsSetProperty90are accepted no-ops;AdsFindConnection25/AdsGetTableHandle25report not-found (OpenADS keys by handle, not path / name). - M12.23 — close the export gap the X# Advantage RDD relies on.
Live run of X#'s
AXDBFCDXRDD against OpenADS'ace64.dllsurfaced ~45 more entry pointsADSRDD.prgbinds by name (AdsGetMemoBlockSize,AdsGetTableOpenOptions,AdsGetBookmark,AdsCancelUpdate,AdsSetField/AdsSetEmpty/AdsSetNull/AdsSetShort/AdsSetMoney/AdsSetTime/AdsSetTimeStamp,AdsGetDate,AdsContinue,AdsEval*Expr, RI / unique / autoinc enforcement toggles,AdsStmt*helpers, …). Forwards where one fits, accept-and-ignore for session / statement toggles,AE_FUNCTION_NOT_AVAILABLEfor the genuinely-unimplemented (so the X# runtime falls back to its own client path). The field-setter family handles the ACE "field name or 1-based ordinal cast to a pointer" idiom. AdsAppendRecordauto-locks the new record (ACE semantics for non-exclusive tables — X#'sGoHotrefuses to write a record it sees as unlocked).AdsIsRecordLocked/AdsLockRecord/AdsUnlockRecordhonourrecno == 0= current record and report the real lock state instead of stubbing 0.AdsCreateIndex61/AdsCreateIndex90option-bit fix: the "descending" flag isADS_DESCENDING(0x08), not0x02—0x02isADS_COMPOUND, which X#'s ADSRDD always sets for CDX orders, so the old mask built every X# order descending andDbGoToplanded on the last key.AdsCreateTable/AdsCreateTable90stage an empty.fptnext to the.dbfwhen the field list has anMfield (usingusMemoBlockSize, default 64) — without itConnection::open_tablecan't attach a memo store and any memo write fails "memo store not attached".- X# RDD against a remote OpenADS server. Three remote-path
fixes so X#'s ADSRDD drives
openads_serverdover the wire (AdsConnect60("tcp://host:port/<datadir>", ADS_REMOTE_SERVER) → AX_SetConnectionHandle → DbUseArea):remote_field_indexnow honours the "field name OR 1-based ordinal cast to a pointer" idiom (X#'s_FieldSubcallsAdsGetFieldType/Length/Decimalsby ordinal — a tiny pointer value was being dereferenced as a string); the remoteAdsOpenTablebranch defaults a missing extension to.dbf; andAdsGetTableFilenamegained a remote path (returning the opened name) instead of failingAE_INTERNAL_ERROR. - Full X#
AXDBFCDXsmoke (tests/smoke/xsharp/AdsSmoke.prg+AdsSmoke_remote.prg) passes end-to-end against OpenADS'ace64.dll. - Test layout. Third-party RDD smoke harnesses under
tests/smoke/(Harbour + X#). GUI showcases (FiveWin, X# WinForms) underexamples/. All opt-in — none run in defaultctest. Doctest coverageabi_versioned_overloads_test.cpp+abi_remote_overloads_test.cpp(gated onOPENADS_TEST_REMOTE). - Clipper-convention empty / past-end / Limbo states.
goto_record(0)is no-op + Eof (not error 5000); empty table reports BOF / EOF +RecNo() = LastRec() + 1; GO past-end enters Limbo; CDX dup-tag silent reopen; CDX dup-key insertion uses recno tie-break. - Index cursor consistency.
goto_recordre-syncs the index cursor to the row's key (so the next SKIP walks the right neighbour); CDX cursor state tracking; hard-seek miss parks the cursor on the>neighbour for SKIP; hard-seek past every key parks at AfterEnd;GO 0keeps Limbo. SET DELETED ONeverywhere. Index walks skip deleted rows; natural-order GOTOP / GOBOTTOM skip deleted; all-deleted underSET DELETEDreports Limbo (not Eof);GOTOP/GOBOTTOMon an empty index report Limbo.- DESCEND wired through.
bFindLastretry on DESCEND retired;DBSEEK( '' )withbSoftSeeklands at the first record +FOUND = .T.; empty-key shortcut applies only to ASC orders. - CDX FOR-clause filter.
CREATE INDEX … FOR <expr>honoured at build time and on every subsequent insert;CREATE INDEXinserts deleted rows too (DBFCDX semantics); each newCREATE INDEXreplaces the active order (Clipper convention); re-CREATE INDEXwith an existing tag clears the old B+tree first;ADS_DOUBLEKEYASCII conversion + creation-order tag ordinals. - rddads compat patches (
tools/harbour_patch/).adsSeekcarve-out:Seek( '' )with the soft-seek flag now leavesfBofalone under Limbo so Harbour's own EOF logic doesn't snap to recno 0;ORDSETFOCUS(N)uses CDX-file insertion order, not handle ids;rddads-compatdefault connection +CREATE INDEXgoto-top. - SKIP overshoot. Cursor stays on the last live record, not recno 0.
- M12.17 —
RemoteTablerow cache. NewFetchCurrentRowopcode returns the entire current record's column values in one frame.AdsGetField/AdsGetLong/AdsGetDouble/AdsGetJulianserve every cell from the cache; W cells per row collapse to 1 RTT. - M12.18 — piggyback row trailer on nav acks.
GotoTopAck/GotoBottomAck/SkipAck/GotoRecordAck/FetchCurrentRowAckcarry the full row buffer + recno + deleted flag.AdsGetField/AdsGetRecordNum/AdsIsRecordDeletedall hit the cache populated by the prior nav ack — zero extra RTT after every nav. - M12.19 — record_count cache.
AdsGetRecordCountandAdsGetRelKeyPosnow serve from a per-table cache, invalidated only on writes that change row count (AdsAppendRecord,AdsDeleteRecord,AdsRecallRecord,AdsPackTable,AdsZapTable). - M12.20 —
TCP_NODELAYdisabled Nagle on every per-connection socket. The OpenADS wire is strict ping-pong, so Nagle's accumulation delay (up to 200 ms) was pure latency tax. Removes ~40–200 ms per RTT on slow links. - Net xbrowse PgDn: pre-M12.17 ~300 RTT → ~20 RTT × ~5 ms (Nagle off) ≈ ~100 ms. ~30× end-to-end speedup vs pre-M12.17.
The wire layer now carries every common ABI op rddads emits when an
app connects via AdsConnect60("tcp://host:port/dir"). Until rc16
most ABI calls past AdsOpenTable collapsed with "unknown table"
because they only resolved local Table*. Fixed across 40+ entry
points:
- M12.14 — field metadata + cursor state.
AdsGetNumFields,AdsGetFieldName,AdsGetFieldType,AdsGetFieldLength,AdsGetFieldDecimals,AdsAtBOF,AdsGetRecordNum,AdsIsRecordDeleted,AdsGotoBottom. NewDescribeTableopcode returns schema in one round-trip (cached onRemoteTablesoadsOpenfield-iter stays cheap). - M12.14b — typed reads.
AdsGetLong,AdsGetDouble,AdsGetJulianreuseGetField, parse client-side. No new opcode. - M12.15 — info / lock / maintenance / AOF.
AdsIsFound,AdsRefreshRecord,AdsGetTableType,AdsGetRecordLength,AdsGetNumIndexes,AdsGetLastAutoinc,AdsLockRecord/Unlock,AdsLockTable/Unlock,AdsPackTable,AdsZapTable,AdsFlushFileBuffers,AdsCloseAllIndexes,AdsSetAOF,AdsClearAOF,AdsGetAOFOptLevel. - M12.15b — memo.
AdsGetMemoLength,AdsBinaryToFile,AdsFileToBinaryreuseGetField/SetField. - M12.16 — index handle subsystem. New
HandleKind::RemoteIndex+RemoteIndexwrapper.AdsOpenIndex/AdsCloseIndex/AdsSeek/AdsSeekLastover the wire. Server lazy-promotes ABI handles intbls_hand syncs the engine cursor after Seek so the two cursors never drift. - M12.16b — remaining index ops.
AdsCreateIndex/AdsCreateIndex61(CDX-on-the-wire),AdsSkipUnique,AdsSetScope,AdsClearScope. - M12.16c — order switching. New ABI exports
AdsSetIndexOrder(by tag name) andAdsSetIndexOrderByHandle(by hIndex). Wire bridges viaSetOrder/SetOrderByNameopcodes.
AdsGetRelKeyPos/AdsSetRelKeyPoshonour the active index walk. Whent->order()->index()is bound, both walk the index once (seek_first → next() loop) to compute / position by key fraction, not raw recno. Cursor recno is restored after theGetprobe so it doesn't visibly move the user-facing position. No- active-order behaviour unchanged.
+(VFP autoincrement) field type. dBASE Level 7 / VFP autoincrement column carries the field-descriptor type byte+; classifier insrc/drivers/dbf_common.cppnow maps it toDbfFieldType::Integer. Prior fall-through left the schema parser atUnknown.
- Windows Service support.
openads_serverd --install-serviceregisters aSERVICE_AUTO_STARTWin32 service;--uninstall-servicedrops the registration; the same binary doubles as both interactive CLI and SCM-driven service through a realRegisterServiceCtrlHandlerwithSERVICE_CONTROL_STOP/SERVICE_CONTROL_SHUTDOWNdriving the same graceful-shutdown path as interactive Ctrl+C. - Linux systemd unit (
scripts/openads-serverd.service) hardened —User=openads,ProtectSystem=strict,NoNewPrivileges,RestrictAddressFamilies=AF_INET AF_INET6,PrivateTmp,Restart=on-failure. - macOS launchd plist
(
scripts/com.openads.serverd.plist) — KeepAlive on crash; stdout / stderr to/var/log/openads-serverd.{out,err}.log.
- M-AOF.6 — production-CDX auto-open in
AdsOpenTable. Opening<base>.dbfauto-binds the sibling<base>.cdxso every tag in it becomes navigable on the Table without an explicitAdsOpenIndex60call. rc12 didn't do that — Studio's per-request short-livedAbiSessionre-opened the DBF on every/rowsfetch but never picked up a CDX created in a prior session, soAdsGetAOFOptLevelreportedNONEforever even afterCREATE INDEX. - Studio — guided AOF demo. Browse-tab
▶ Demobutton walks the full Rushmore-style AOF story end-to-end against whatever table is active. - Studio — AOF hint chips functional. When the AOF doesn't
reach
FULL, Studio surfaces a chip per character / memo field referenced by the cond that doesn't have a matching index yet. Click → runsCREATE INDEX <field>_IDX ON <table> (<field>)+ re-applies the AOF. openads_serverd --versionnow reports the actual tag viagit describe --tags --always --dirtyat CMake configure time. Previously hard-coded1.0.0-rc1since rc1.
First working slice of Advantage Optimised Filters (AOF) —
Rushmore-style query optimisation. AdsSetAOF parses + evaluates
the cond, installs a per-record bitmap as a filter predicate that
Skip / GoTop honour, and routes individual leaves through CDX /
NTX index range scans whenever an open index has the leaf's field
as its key expression. AdsGetAOFOptLevel reports
ADS_OPTIMIZED_FULL / PART / NONE based on per-leaf coverage.
Sparse-bitmap navigation lifts the visible-set walk from O(N) to
O(M).
V1 grammar (identifiers + keywords case-insensitive, both
Clipper-style .T. / .AND. and SQL-style accepted):
<field> OP <literal> OP in { = == != <> # < <= > >= }
<field> BETWEEN a AND b
<field> IN ( v1, v2, ... )
expr AND expr OR NOT expr ( expr )
Index-accelerated leaves in V1: character / memo fields with a
bare-field-name index expression; operators Eq, Ne, Lt, Le, Gt, Ge,
Between, In. Numeric / date / logical fields, UPPER(field) /
compound expressions still produce a correct bitmap via per-record
fallback (don't count as "served by index").
- Studio mode badge. SPA header now shows 🏠
LocalServer(green) when the console runs in-process insideace64.dll/ace32.dll, or 🌐Remote Server(blue) when hosted byopenads_serverd. Hover reveals the active data directory. Signal from a newmodefield on/api/health("localserver"whenHttpConsolewas started without a backing wire-server pointer,"remote-server"otherwise). Reverse-proxy deployments that strip unknown fields keep working unchanged — the badge stays hidden when/api/healthis unreachable.
Studio web console is now embedded inside ace64.dll / ace32.dll
itself. A LocalServer application — Harbour / X# / Clipper loading
the OpenADS DLL directly without launching openads_serverd.exe —
spins up the console in its own process.
- Three OpenADS-only entry points:
AdsStudioStart(port, data_dir),AdsStudioStop(),AdsStudioPort(&port)(ordinals 238–240). - Env-driven auto-start:
OPENADS_STUDIO_PORT=<port>before launching the host app;OPENADS_STUDIO_DATA/OPENADS_STUDIO_HOSTdefault to"."/127.0.0.1. Without the port env var the auto hook is a no-op — no surprise localhost listener. - Compiled into the DLL only when
-DOPENADS_WITH_HTTP=ON(default since rc20). Without that flag the three exports returnAE_FUNCTION_NOT_AVAILABLEso callers can detect the build flavour at runtime.
Addresses XSharp-Project feedback (Robert van der Hulst):
- Dual x64+x86 ZIP. Bundle ships both
ace64.dllandace32.dllplus matchingopenads_serverd_{x64,x86}.exe/openads_bench_{x64,x86}.exe. X#, Harbour-x86, and legacy Clipper apps all pick the right bitness without a separate download. - Static-linked mbedtls 3.6.2. Zero runtime
libssl/libcrypto/mbedtlsDLL dependency. Verified viadumpbin /dependents: only KERNEL32, WS2_32, bcrypt, MSVCP140, VCRUNTIME, and Windows ucrtapi-ms-win-crt-*show up. openads_serverd --port <N>plus an explicit "port 6262 is the SAP Advantage Database Server default" hint when the bind clashes with a running ADS service.tools/bench/README.mddocuments thatopenads_benchsynthesises its own three-column DBF (ID N(8,0),TAG C(4),AMT N(8,2)) on each run from a fixed seed; nothing is shipped, fully reproducible.
Studio schema view shows the ADS field-type letter (C / N / M / D / L / I / Y / B / V / Q / G) instead of the numeric code, with the ADS type-letter mapping corrected; the numeric tooltip is dropped.
Studio binary-memo serializer + Win x86 build fix. Binary memo
cells are now base64-encoded in /api/tables/<t>/rows so JSON
round-trips cleanly through the SPA.
CDX leaf+branch full multi-level split + memo > 64 K fix +
cross-platform -Werror clean.
M(cdx-split)— full multi-level B+tree leaf+branch split. Stress harness now exercises 2 bags × 2 tags each at 200 K rows;CREATE INDEXdefaults back to CDX (the previous NTX fallback is retired).- DBF compat. Accept 0xF5 / 0xFB headers + single-letter type
codes;
AdsGetFieldno longer truncates memos > 64 K to 65534 bytes. - Cross-platform.
ace32.lib/ace64.libimport libs + ordinal-compat tooling shipped. GCC 13 (Ubuntu 24.04) + clang Release pass-Werrorcleanly:-Wshadow,-Wstringop-truncation,-Wformat-truncation,-Wsign-conversioninjulian_to_ymd,COL%zufield-name copy, stress harnesses,probe_cdx2, drivers/cdx iterators. - Studio.
studio.web.0.12ZIP backup of data dir,studio.web.0.13table-type override + memo hex viewer,studio.web.0.14host OS / arch / compiler banner.
Studio sessions kill + JSON export + TLS deployment docs.
- Studio.
studio.web.0.10bulk select + saved queries + SQL highlight;studio.web.0.11kill-session button + JSON export. - TLS deployment guide (EN / ES / PT) — proxy, stunnel, SSH tunnel recipes for fronting the cleartext server with TLS until M12.12 ships.
- CI release-workflow build timeout bumped 45 → 60 minutes.
Studio web 0.4 – 0.10: Sessions, Data Dictionary tab + REST,
Reindex / Pack / Zap + CREATE INDEX wizard + memo viewer, sidecar
list / server stats / DBF upload, HTTP Basic auth + table download
- theme toggle, Browse sort + filter + i18n (EN / ES / PT). Data
Dictionary documentation pages (EN / ES / PT).
b64decodesign-conversion fix.
Studio web console + bilingual+ documentation site + release CI.
- OpenADS Studio (
studio.web.0.1…0.3) — clean-room ARS-equivalent web console embedded inserverd. CRUD + paginated browse + server info;CREATE/DROP/ encrypt + SQL history. All third-party-product-name references scrubbed from Studio copy.AdsGetLastErrorcaptured before close + 'did you mean' hint on connection failures. Docs link in header. - Documentation site — bilingual+ EN / ES / PT under
docs/, published through GitHub Pages workflow. Benchmarks pages + Studio screenshots. - Release CI —
releaseworkflow builds + packages + publishes on tag push.docs/superpowersandbuild/are export-ignored from source archives.
openads_bench v2 + CI matrix gains a TLS=ON entry.
Real TLS transport, transport abstraction, wire-protocol spec.
- M12.12 — real TLS via vendored mbedtls 3.6 LTS (Apache-2.0).
The
tls://URI scheme reserved in 0.3.6 now wires through to a real handshake on both client and server. - M12.13 — transport abstraction. The wire protocol no longer
bakes in the socket type; cleartext, TLS, and any future
transport plug in through a single virtual interface.
docs/wire-protocol.mddocuments the wire format. - CI runners + actions bumped to current versions; missing
<tuple>include added.
tls:// URI scheme reserved (M12.12 stub) ahead of the real
handshake landing in 0.4.0.
Wire-protocol semantics complete: ACE error-code propagation
(M12.10) and batched row fetch (M12.11 — Fetch / FetchAck).
Phase 2 server feature-complete (read + write + SQL + index + auth).
- M12.6 — remote write surface (
append/set/delete/recall/goto/flush). - M12.7 — remote SQL exec (
ExecuteSQLwire op). - M12.8 — remote
AdsReindex. - M12.9 — server-side authentication.
RemoteConnectiondtor now callsdisconnect.
Phase 2 server alive end-to-end + cross-platform SQL bench.
- M12.3 — server accept loop +
Hello/Connectdispatch. - M12.4 — remote read-only table ops.
- M12.5 — dual-mode DLL:
tcp://URIs route the ABI to the server; local-mode paths still hit the in-process engine. openads_serverdstandalone TCP server CLI.openads_benchcross-platform SQL workload timer.- macOS bring-up:
setup_macos.shone-shot script,HOST_NAME_MAXfallback,accept()self-connect wake-up, same-process lock contention test skip. - Linux bring-up: OFD locks for fd-scope contention,
shutdown(SHUT_RDWR)beforeclose()so blockedaccept()wakes, third + fourth waves of clang-Werrorsign / include / unused fixes,-Wreturn-type-c-linkagecleanup on internal helpers.
SQL window-function + scalar-fn deepening + sockets layer.
- M10.49 PARTITION BY.
- M10.50 RANK / DENSE_RANK.
- M10.51 qualified column refs.
- M10.52 multi-row
VALUES. - M10.53
NULLIF/COALESCE/IFNULL. - M10.54
FILTER (WHERE …)aggregate clause. - M12.2 sockets layer (cross-platform).
SQL date / CTE / NULL surface + collation + wire skeleton.
- M10.43 multi-arg scalar fns.
- M10.44
IS NULL/IS NOT NULL. - M10.45 date scalar fns.
- M10.46 derived tables.
- M10.47
ROW_NUMBER(). - M10.48 CTE (
WITH …). - M11.6 NULL bitmap.
- M11.7 collation.
- M11.8 OEM / UTF-8 conversion.
- M12.1 wire skeleton.
SQL maturity wave: every JOIN flavour, the full subquery / aggregate / DISTINCT / UNION / CASE / LIMIT surface, plus the AEP host, AES-256-CTR encrypted DBFs, VFP V / Q field types, and nested transactions. Brings the SQL layer to "real apps run through it" status.
- JOIN matrix.
INNER(M10.13 parse + M10.14 executor),LEFT OUTER(M10.16),RIGHT OUTER(M10.21),FULL OUTER(M10.22 — union of LEFT + RIGHT),JOIN+WHERE/ORDER BYcombos (M10.20),JOIN+ aggregate combo (M10.23),GROUP BYacrossJOIN(M10.34). - Subqueries.
INliteral lists + subqueries (M10.15),EXISTSuncorrelated (M10.17), correlatedEXISTS(M10.24), scalar subquery<col> op (SELECT col FROM t)(M10.18), aggregate scalar subquery (M10.19), correlated scalar subquery (M10.29), correlatedINsubquery (M10.35). - Aggregates + grouping.
COUNT(*)/COUNT/SUM/AVG/MIN/MAX(M10.10),GROUP BY+HAVING(M10.25), HAVING expression tree (M10.30), aggregate /GROUP BYinsideUNIONmembers (M10.36). - Set + shape ops.
UNION/UNION ALL(M10.26),UNION+ projection (M10.27),UNION+ORDER BY(M10.28),DISTINCT(M10.31),LIMIT/OFFSET(M10.32),BETWEEN/LIKE(M10.33), multi-columnORDER BY(M10.37),CASE WHENin projection (M10.38). - DML / DDL.
INSERT(M10.5),ORDER BYexecution (M10.6),UPDATE/DELETEbulk throughAdsExecuteSQLDirect(M10.7), projection lists (M10.8), DDLCREATE TABLE/CREATE INDEX(M10.9), VFP autoinc fields with persistent counter (M10.11),AdsRestructureTableCHANGE (same-type length / decimals) (M10.12),INSERT INTO … SELECT(M10.41),CREATE TABLE AS SELECT(M10.42). - Other SQL.
WHEREOR/NOT/ parens + numeric literals (M10.3), scalar string fns (M10.39), arithmetic in projection (M10.40). - Engine + drivers. Real
.addData Dictionary persistence — round-trips through.addreopen (M10.1); VFP I / Y / B field decode + encode (M10.2); branch-level MISS closure (AdsRestructureTableDELETE-fields +AdsSetIndexDirection) (M10.4). - AEP host (M11.4).
CREATE PROCEDURE+EXECUTE PROCEDUREthrough the OpenADS clean-room AEP runtime. - Encrypted DBFs (M11.2). OpenADS-encrypted DBF — AES-256-CTR per-page, transparent through the read / write path.
- VFP V / Q (M11.1). Varchar + Varbinary field types.
- Nested transactions (M11.3).
AdsReleaseSavepoint+ nestedBEGIN/COMMIT(proper save-point stack).
The 0.2.0 release closes the entire 226-symbol Harbour-reachable
Ads* ABI surface — every export resolves to either a real
implementation or a documented local-mode silent-success. No exports
hard-fail with AE_FUNCTION_NOT_AVAILABLE at the function level any
more. The release also relicensed the project from MIT to Apache
License 2.0 and added a clean-room provenance / non-commercial /
no-warranty disclaimer block + NOTICE file.
- Compound CDX expression evaluator.
UPPER,LOWER,LTRIM/RTRIM/ALLTRIM,STR(n[,len[,dec]]),DTOS(date),SUBSTR(s,start[,len]), and string concatenation with+. UPPER / LOWER / SUBSTR walk UTF-8 codepoints (ASCII + Latin-1 supplement case map,ÿ↔Ÿpair); the Latin-1 case mapping table closes the M9.17*WUnicode surface. - Real CRUD for tables and indexes.
AdsCreateTableparses the rddadsNAME,Type,Len,Dec;…field-def syntax;AdsCreateIndex61/AdsCreateIndexbuild CDX or NTX bags compatible with FoxPro and Clipper layouts.AdsZapTable/AdsPackTable/AdsReindexmatch the Clipper bound-index lifecycle. - Multi-file index binding. Multiple
.ntxfiles (or multiple pre-built.cdxbags) coexist on a single Table; same-path reopen refreshes;AdsCloseIndexdrops the closed view without disturbing the active order. - Transactions + savepoints + WAL recovery.
AdsBeginTransaction/AdsCommitTransaction/AdsRollbackTransaction/AdsCreateSavepoint/AdsRollbackTransaction80(savepoint). Mid- tx crash + reopen replays the WAL and writes back before-images for orphan transactions. - Memo (DBT / FPT) read + write + binary type. Text memos
round-trip through DBT and FPT;
AdsGetBinary/AdsGetBinaryLength/AdsSetBinarycarry binary blobs through FPT block-type tags (Text / Picture / Object); chunkedAdsSetBinarywrites reassemble through a per-(table, field) accumulator. - Locking with retry policy.
AdsLockTable/AdsLockRecorduse non-blocking byte-range acquires (LockFileEx LOCKFILE_FAIL_IMMEDIATELYon Windows,fcntl F_SETLKon POSIX);AdsSetLockCycle/AdsSetLockRetryCountconfigure the retry budget. - Full-text search.
AdsCreateFTSIndexwrites a clean-room# OpenADS FTS v0text file per table;AdsFTSSearchand the SQLCONTAINS(<col>, '<query>')predicate intersect per-token recno lists with AND semantics. - Server / dictionary surface.
AdsMg*(15 calls) report local- mode "everything quiescent" responses;AdsDD*(14 advanced-DD calls) accept silently and zero-fill property getters. Real persistence in the OpenADS DD format lands with 0.3.x. - Schema evolution.
AdsRestructureTableADD-fields path rewrites the DBF with extended schema and preserves every record's original-field bytes; DELETE / CHANGE arguments still surface AE_FUNCTION_NOT_AVAILABLE pending VFP / ADT structural extensions. - Misc. Real
AdsGetServerName/AdsGetServerTime,AdsGetLongLong,AdsSetFieldRaw,AdsVerifySQL,AdsFailedTransactionRecovery,AdsGetAllLocks,AdsSkipUnique,AdsFindFirstTable/AdsFindNextTable/AdsFindClose,AdsCopyTable/AdsCopyTableContents/AdsConvertTable,AdsAddCustomKey/AdsDeleteCustomKey.
- License: relicensed MIT → Apache License 2.0 (
LICENSE+NOTICE). - Independence + non-commercial purpose + clean-room provenance + no-warranty + downstream responsibility — block added to the README and mirrored to the NOTICE file (Apache 4(d) preservation).
- Tests: 214 doctest cases, 3865 assertions, all green on
Windows / MSVC Release. CI matrix builds Windows + Linux + macOS
cleanly through
.github/workflows/ci.yml.
| Tag | Milestone |
|---|---|
m9.1 |
Compound CDX expression evaluator |
m9.2 |
Stub batch reorganised into real / no-op / missing |
m9.3 |
Compound expressions validated through Harbour |
m9.4 |
AdsGotoRecord + table / file metadata |
m9.5 |
AdsCreateTable |
m9.6 |
AdsRefreshRecord + AdsExtractKey |
m9.7 |
AdsCreateIndex61 with compound expression |
m9.8 |
AdsZapTable + AdsPackTable |
m9.9 |
AdsReindex |
m9.10 |
NTX multi-level B+tree split |
m9.11 |
AdsCopyTable / AdsCopyTableContents / AdsConvertTable |
m9.12 |
AdsFindFirstTable / AdsFindNextTable / AdsFindClose |
m9.13 |
Binary memo (AdsGetBinary / AdsSetBinary / AdsGetBinaryLength) |
m9.14 |
NTX multi-tag binding |
m9.15 |
Real AdsGetServerName / AdsGetServerTime + binding-leak fix |
m9.16 |
Chunked AdsSetBinary |
m9.17 |
Unicode *W variants |
m9.18 |
Lock retry / cycle policy |
m9.19 |
AdsCreateFTSIndex |
m9.20 |
AdsAddCustomKey / AdsDeleteCustomKey |
m9.21 |
FTS search side (AdsFTSSearch + SQL CONTAINS) |
m9.22 |
UTF-8 codepoint-aware index-expression evaluator |
m9.23 |
Misc MISS fillers (LongLong / FieldRaw / VerifySQL / FailedTxRecovery / GetAllLocks / SkipUnique) |
m9.24 |
Local-mode AdsMg* surface (15 calls) |
m9.25 |
Local-mode AdsDD* CRUD surface (14 calls) |
m9.26 |
AdsRestructureTable (ADD-fields path) |
m9.27 |
CI matrix portability |
Final 0.1.0. The post-rc1 work below extends the Harbour smoke
beyond the read path covered in 0.1.0-rc1: a real Harbour app now
also drives multi-tag focus swaps, ARIES-style transactions, and
memo M-field round-trips end-to-end through OpenADS' ace64.dll.
AdsOpenIndexwidened to its real 4-arg signature(hTable, pucName, ahIndex[], &pu16ArrayLen). Every tag inside a compound CDX is opened by name throughCdxIndex::open_named; the first tag's IIndex moves intoTable::set_orderand the rest park in their bindings.Table::take_order()/Order::release()surrender the active index'sunique_ptr<IIndex>so a focus swap can park it in the previous binding's slot.get_tableandtable_for_indexnow callactivate_binding(h)whenever a navigation call arrives with an index handle, so rddads'pArea->hOrdCurrentswaps drive the Table's active order in lockstep.AdsGetIndexHandlestrips trailing whitespace from the caller's tag name;AdsGetIndexName/AdsGetIndexExprread each binding's metadata directly so parked tags report their real name even before they become live.AdsGetNumIndexesreturns the per-table binding count.
- A real Harbour app drives
AdsBeginTransaction/AdsRollback/AdsCommitTransactiondirectly. BEGIN + APPEND + ROLLBACK leaves the appended row in the DBF flagged deleted (CDX index entries persist by design —Found()still reportsTbutDeleted()isT); BEGIN + APPEND + COMMIT persists durably to both the DBF and every CDX tag. Table::register_extra_index_view/Table::unregister_extra_index_view/Table::clear_extra_index_viewstrack the parked CDX sub-tags as non-owning views; the binding still owns the IIndex lifetime.Table::snapshot_index_keys_()captures the pre-write key per index — active order plus extras — andsync_all_indexes_(snap)erases each prior(recno, prev_key)and inserts the new one in lockstep, so aset_fieldon a multi-tag CDX keeps every tag consistent (M8.8 only synced the active order).Table::flush()flushes the active order and every extra view so a multi-tag commit reaches disk for every tag.
- A real Harbour app appends rows whose
FIELD->NOTEScarries a short memo (43 bytes) and a longer memo (280 bytes), closes the area, reopens, and reads the memos back via the standard Clipper RDD surface. make_cdx.exenow also writes an emptydata.fptnext todata.cdxviaFptMemo::create, soConnection::open_tablefinds a memo store to auto-attach when the DBF declares an M field.AdsGetMemoLength/AdsGetMemoDataType/AdsGetStringare now real implementations usingresolve_field_index(M4 had earlier versions that only accepted string field names; rddads passes theADSFIELD(n)integer form).AdsCloseTableflushes the table before releasing the handle so non-transactional appends reach disk onUSEclose.ADS_MEMO_TEXT/ADS_MEMO_PICTUREaliases resolve to the M8.4-verifiedADS_STRING(4) andADS_IMAGE(7) values.
First end-to-end validation against Harbour contrib/rddads. A real
.prg compiled with hbmk2 -comp=msvc64 -lrddads -lace64 opens a
DBF, walks records, runs dbSeek, appends rows, and reopens — every
call lands on OpenADS' ace64.dll with no Harbour rebuild.
- 5-layer architecture: ABI shim → Session/Connection → SQL → Engine (Table / Index / MemoStore / LockMgr / TxLog) → OS abstraction.
- DBF read/write for C / N / L / D columns; deletion flag; flush.
- CDX driver with compound layout (file header + structure-tag root +
per-tag CDXTAGHEADER + sub-tag B+tree). Multi-tag-per-file API
(
add_tag/open_named/list_tags). FoxPro-equivalent leaf bit-pack (mirrors Harbourhb_cdxPageLeafInitSpace). Compound CDX closes the last reviewer-flagged compat-breaking item. - NTX driver with cache-based in-order traversal for multi-level trees; leaf-split fix promotes the separator without duplicating it.
- AES-128 / AES-256 ECB primitives (vendored tiny-AES-c, Unlicense), validated against FIPS-197 + NIST SP 800-38A.
- DBT + FPT memo round-trip.
- Data Dictionary (
.add):TABLE alias=pathtext format with alias resolution. - Minimal SQL:
SELECT * FROM <table> [WHERE col op 'lit' [AND ...]]with six comparison operators.
- OS byte-range locking with ranges compatible with original ACE so installs can coexist during migration.
- WAL with CRC-32C records (BEGIN / UPDATE / COMMIT / ABORT).
- In-memory ordered op log with named savepoints (M5.3).
- Group commit (M5.4): each record carries a monotonic LSN;
sync_to(lsn)is the group-commit primitive — first thread to observelast_synced_lsn_ < lsnissues a single fsync covering the high-water mark. - Idempotent recovery (M5.5) via
openads.lsnmapsidecar. Concurrent recovery passes can never regress the per-record watermark.
.addparser,Connection::open(<path>.add)resolves member tables through the dictionary on everyAdsOpenTable.
- Parser + executor for
SELECT *+ multi-clauseWHEREjoined by implicitAND. Compiles to aTable::RowPredicateclosure used byAdsExecuteSQLDirectto filter the cursor's record stream.
- M8.0
ace64.dll/ace32.dllSHARED CMake target with a.defexporting 80 realAds*entry points. - M8.1 226
Ads*exports — superset of every symbol Harbourrddads.libreferences; the 146 newly-stubbed entries returnAE_FUNCTION_NOT_AVAILABLE(5004) so the link resolves cleanly. - M8.2 Six legacy MSVC2013-era CRT shims (
_dclass,_dsign,_wfsopen,_getch,_kbhit,_eof) re-exported under aliases so Harbour's prebuilt msvc64 libs link against modern UCRT. - M8.2
smoke.exeruns end-to-end:AdsVersion()resolves through the rddads wrapper to OpenADS'AdsGetVersion. - M8.3
USE data VIA "ADSCDX"+ walk records.Connection::open_tableauto-appends.dbf.AdsGetField/AdsGetFieldType/AdsGetFieldLengthaccept either string field names or theADSFIELD(n)integer-cast-as-pointer form.AdsConnectis now a real wrapper aroundAdsConnect60. - M8.4 ACE field-type constants verified by sweeping
AdsGetFieldType's return through 0..40 against rddads. Result:ADS_LOGICAL = 1,ADS_NUMERIC = 2,ADS_DATE = 3,ADS_STRING = 4, ... — the inverse of the public ACE SDK ordering in some places. Mapping captured ininclude/openads/ace.h. - M8.5 Multi-field DBF (C/N/L/D) end-to-end.
AdsGetFieldDecimals,AdsGetLong,AdsGetDouble,AdsGetJulianreal impls (was 5004 stubs);AdsGetJulianparsesYYYYMMDDand computes Clipper Julian Day Numbers using the same Gregorian formula ashb_dateEncode. - M8.6
dbSeekend-to-end through OpenADS' CDX.Table::path(), index path resolution + auto-extension, polymorphicget_table(accepts table or index handles — rddads'adsGoTopcallsAdsGotoTop(hOrdCurrent)when an order is active), 6-argAdsSeeksignature matching rddads, realAdsIsFoundreadingTable::last_seek_found_. - M8.7 Write path:
dbAppend+FIELD-> := value+dbCommit+ reopen.AdsSetString/AdsSetLogical/AdsSetDouble/AdsSetLongLong/AdsSetJulianreal impls; field index resolution viaresolve_field_index. - M8.8 Active index auto-syncs on every record mutation.
Table::compute_index_key_evaluates bare-field-name expressions against the currentrecord_buf_;Table::sync_active_index_erases the prior(recno, prev_key)and inserts the new one.Table::flush()flushes both the driver and the index.
- 135 doctest cases / 1820 assertions passing on Windows / MSVC Release.
- One
tests/harbour_smoke/integration harness producing a runnablesmoke.exethat exercises the full Harbour → rddads.lib → OpenADS path.