[2/3] Integrate XSK maps into the core XDP datapath

[ProjectsByJackHe]

Description

Full E2E demo using tools from this PR : DEMO

Part 2 of the plan: #5982
Fixes #5972

Adds the core msquic datapath integrations for the new API contract defined in #5983

Key design decisions:

• If map mode is enabled, then we do not create the WinSock or Epoll or .... datapaths. No more best effort!
• Init failures in map mode hard fails the datapath initialization. No silent failures!

Testing

Added datapath unit tests. See the E2E demo.

Additionally, the plan is to leverage some supporting PRs that included code to ingest the latest XDP version, and kicking off a couple of manual CI runs based on that: https://github.com/microsoft/msquic/actions/runs/27313685557/job/80690420168

Documentation

Will come as part 3 in the plan: #5982

[codecov]

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.91%. Comparing base (ada1bbf) to head (96eaab7).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/core/settings.c	33.33%	2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (60.00%) is below the target coverage (80.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6003      +/-   ##
==========================================
- Coverage   85.96%   84.91%   -1.05%     
==========================================
  Files          60       60              
  Lines       18847    18852       +5     
==========================================
- Hits        16202    16009     -193     
- Misses       2645     2843     +198     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ProjectsByJackHe]

Planning on waiting until #6006 gets resolved to address the next wave of feedback.

[mtfriesen]

It's not clear to me why we don't have end-to-end tests exercising everything from providing the XSKMAP before creating a registration through actual XDP data paths flowing? Since we intend to officially support this feature, it is risky for both us and our partners to supply/depend on an API that does not have regression test coverage and a paved pattern to follow.

[ProjectsByJackHe]

It's not clear to me why we don't have end-to-end tests exercising everything from providing the XSKMAP before creating a registration through actual XDP data paths flowing? Since we intend to officially support this feature, it is risky for both us and our partners to supply/depend on an API that does not have regression test coverage and a paved pattern to follow.

That's in the works. I took the approach of building up an E2E flow manually first to see the shape of how the datapath integration would look like. There's likely going to be some limitations in the CI environment that prevents some scenarios from being exercised, but I am currently adding coverage for the parts that can be automated now that the skeleton of the general changes is mapped out in the current PR iteration.

[mtfriesen]

Let's ensure we add tests with each chunk of work we do. I understand it may be a lot of work to add tests, but the best time to protect the code with automation and review the coverage is when making the change.

[ProjectsByJackHe]

Let's ensure we add tests with each chunk of work we do. I understand it may be a lot of work to add tests, but the best time to protect the code with automation and review the coverage is when making the change.

XDP needs to release a new driver version consumable by MsQuic in the CI before I can push my tests.

I have them right now working over a local VM with the latest xdp drivers installed, and find that everything is working.

[ProjectsByJackHe]

Either we merge #6034 (or wait until the next XDP release) to add automated coverage, or having a one-off manual dispatch for the tests + manual local VM runs is good enough to give confidence to merge this integration. Let's decide on this first and align timelines. Currently, the map tests are pushed for review, but disabled.

[ProjectsByJackHe]

Might be wondering, why -1? it was previously 0x1234 (which I thought was out of bounds for windows handles).
Why not NULL or INVALID_HANDLE (0)? Indeed, 0 is invalid on windows but not posix.

Given that the general abstraction of raw-only-datapath mode is cross-platform, POSIX runs these tests too, and so we can add that coverage.

[mtfriesen]

That's a good point. Maybe we need a cross-plat abstraction for "invalid [socket|map|file] handle" which can either be a value we know must be invalid at compile time, or open some dynamically created object (e.g., an event object) that we expect will mismatch the expected handle type.

[mtfriesen]

We need to solve the testing problem - map mode should run all existing MsQuic tests that are not intrinsically incompatible with an XDP map.

Nearly all MsQuic protocol level tests should be compatible with XDP maps, so getting those exercised is a min bar.

[mtfriesen]

Before merging this PR taking a dependency on a prerelease feature, we need to merge the upstream PR that automates prerelease publishing.