Pull from upstream#1
Open
deepakverma wants to merge 499 commits into
Open
Conversation
#2525) Sending tracers is not necessary because we just connected to the nodes a few seconds ago. It causes problems because sent tracers do not have enough time to respond.
Occasionally we'd see `chunkLength` errors from `StringWriter` `.ToString()` calls after connecting. I think we've isolated this (via test stress runs) down to a write happening post-lock on the `TextWriterLogger` disposal. This lock in dispose ensures we're not trying to write to a writer we should have fully released at the end of a `.Connect()`/`.ConnectAsync()` call.
Fixes: - `DisconnectAndNoReconnectThrowsConnectionExceptionAsync`: Under load, we're not connecting in 50ms for the initial thing, and stability > speed, so give a little on this test to help out. Average runtime will be higher, but that's way better than sporadically failing. - `Roles`: Don't assume we're in the initial primary/replica setup (we may have failed over) and make it work either way.
* properly categorizing EXPIRETIME and PEXPIRETIME * release notes
This allows connections through Sentinel via RESP3 by adding `HELLO` to the command map. Fixes #2591.
In Linux, we see 15 minute socket stalls due to OS-level TCP retries. What this means is the PhysicalConnection detects no issues on the pipe that's retrying, but is also not receiving data at all leading to long stalls in client applications. The goal here is to detect that we're timing out commands people have issued to the connection but we're getting _NOTHING_ back on the socket at all. In this case, we should assume the socket is dead and issue a reconnect so that we get out of the hung situation much faster. For an initial go at this, we've chosen 4x the timeout interval as a threshold, but could make this configurable if needed.
* add type-forward on IsExternalInit to resolve #2619
* Simplify InternalsVisibleTo * move PublicKey to Directory.Build.props * docs: add release notes entry --------- Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
* honor select disposition in transactions * rearranging if/switch * using server instead of multiplxer * release notes
Co-authored-by: Nick Craver <nrcraver@gmail.com>
This adds a bit of null ref handling (few ifs). Fixes #2576. Overall, this is biting people in the shutdown race, more likely under load, so let's eat the if checks here to prevent it. I decided to go with the specific approach here as to not affect inlining.
* fix #2642 1: don't treat trivial clusters as clusters - Alibaba uses this config 2: report synchronous failures immidiately and accurately * instead of using node count, use explicit tracking of the DB count * release notes
* fix #2653 * Update src/StackExchange.Redis/ServerEndPoint.cs Co-authored-by: Philo <philon@microsoft.com> * release notes * merge fail --------- Co-authored-by: Philo <philon@microsoft.com>
This is a new feature for allowing keepalive commands to be sent every heartbeat regardless of if they're needed for actual keepalives. I've gotten a ping about a network drop that went undetected for ~8 minutes because of usage only in strings. We see this use case not uncommonly, and because of that string type being the only thing used, we don't detect a protocol fault from subsequent string commands. The symptoms here are partial/unexpected string payloads but ultimately the wrong answers to the wrong query. Given we don't know at what layer this drop happens (this appears to be extremely rare, 1 in quadrillions as far as we know), the best we can currently do is react to it ASAP. There may be a better way to accomplish what this is after - discussing with Marc soon as we're both online but prepping this PR in case it's best path.
-Fix spelling
#2659) * add new AddLibraryNameSuffix API for annotating connections with usage * fixup test * new partial for lib-name bits * use hashing rather than array shenanigans * move to shipped; fix comment typo * comment example * reverse comment owner
* Provide new LoggingTunnel API; this * words * fix PR number * fix file location * save the sln * identify smessage as out-of-band * add .ForAwait() throughout LoggingTunnel * clarify meaning of path parameter
* 1. test and fix incorrect routing of some "sorted set" and "stream" commands 2. fix via new custom efficient message cores 3. add reusable helpers for common null checks on key/value arrays 4. prep work for likely ZDIFFCARD/ZUNIONCARD * rationalize * prefer async tests * add release note
* - fix snafu with NOCASE (logic inverted) - add IsReversed to simplify reversal logic * PR ref * - support ARINFO FULL - refactors ArrayInfo initialization to be span based, and put the main logic inside ArrayInfo technically this is a break, but: `[Experimental]` * release notes
* SENTINEL: defensive check around ROLE * PR number * CI fail; try updating test-reporter
* bare bones INCREX first stab * tidy INCREX impl * add INCREX integration tests * merge * compensate for 8.8RC1 API and behaviour changes * let's try adding the new files * oops, make sure flags is the final arg * account for last-minute-changes to INCREX * add my expectation assertions as integration tests; multiple failures - I'm asking "server" for guidance * update CI image * update test expectations to implement the reality of the server as-at 8.8 * update CI image to 8.8
Propagate exceptions out of Message.WriteTo so PhysicalBridge's outer write path can record a connection failure, and have HandleWriteException kill the PhysicalConnection via RecordConnectionFailed. Without this, a write that throws partway through a frame (e.g. an OOM during serialization) leaves bytes on the wire while the response queue still considers the slot healthy. The next reply from the server then matches against the wrong in-flight message — the symptom seen in #2883, #2804, and #2919, where commands return values intended for a different caller. HighIntegrity mode mitigates the symptom by detecting the desync after the fact via per-message echo checksums; this change addresses the underlying cause for the write-side variant. Adds three tests: - WriteTo must rethrow non-RedisCommandException out of WriteImpl, so the outer bridge catch can act on it. - RedisCommandException continues to surface unwrapped (it carries its own meaning and is excluded from the WriteTo catch filter). - End-to-end: a Message whose WriteImpl throws faults the awaiter with a RedisConnectionException(InternalFailure) AND raises a ConnectionFailed event, proving the physical connection was torn down.
* investigation into socket failure * 1. make TCP keep-alives an explicit option (default: "on") 2. hard-disable TCP keep-alives on DNS endpoints for non-Windows before .NET 10 * release notes * update CI image to 8.8 * fix typo in runtime test isolation / port selection. * Add OptionKeysAreAllNormalized to ensure all keys are correctly normalized; add tcpKeepAlive to docs * stabilize DeleteLastElementPublishesArrayDeleteBeforeKeyDeleteNotifications * allow more grace while waiting for ArrayTests work; only run that test in DEBUG * reduce CI brittleness by restricting tests that have wide side-effects; they can't run in parallel between runtimes * more CI stabilization; HOTKEYS this time
Parenthesize `??` as `+` binds stronger
* WIP v3 (#3035) * WIP v3 merge attempt * fix refs/externs * nits * ascii hash cleanup * remove unused local function * v3 remove pipelines.sockets.unofficial (#3042) * remove Pipelines.Sockets.Unofficial * wip * remove the last traces of Pipelines.Sockets.Unofficial * fix deps * WIP: deps * fix build deps * rev GitHubActionsTestLogger * - fix delegate [UnsafeAccessor] usage - implement Mono pattern * merge (incomplete) * merge * missed a sync-context await * fix missing configureawait usage * missing VT consumption * checking CI fail * versioning * don't test WriteMode.Sync/Async yet * failing test for #3048 * upstream #3049 into v3 * release notes * fix broken config test * re-add channel handling (merge) * optimize pub/sub check for server-defined channels * prefer WriteBulkString("..."u8) => WriteRaw(RESPu8) * optimize CommandMap by pre-generating all the RESP chunks * Move CI to 8.8 rc1 (#3079) * update CI to 8.8 rc1 * and the dockerfile * Draft support for 8.8 arrays (#3076) * draft support for 8.8 arrays * - mark API as experimental - add keyspace notification tests - tidying - docs * clarify how last-items interacts with ring buffers * fix CI netfx compilation * use ValuePairInterleavedProcessorBase to ensure that jagged vs flat doesn't impact us (RESP2 vs RESP3) * make life even easier * add Array to signature prefix list * stabilize hotkeys CI * fix last-minute ARGREP result change * CI: graceful skip if DEBUG not available * Alternative AwaitableMutex implementation for netfx (#3081) * implement AwaitableMutex for netfx * optimize successful sync take path * words * optimize async path for immediate acquisition * re-implement TCP keep-alive * default RESP3 everywhere (#3084) * default RESP3 everywhere * Retain <v6 RESP2 logic; default to v6 * move ship file entries * improve logging for ReadNotificationAsync * stabilize CI for RetryPolicyFailureCases * merge * broken merge (unshipped.txt) * add all the files * build fix * merge drift * CI: allow more time for Echo * use Task machinery for teardown test * Drag benchmark from respite branch * typo * "classic" * BufferedStreamWriter threading tests and fixes * more clarifications on BufferedStreamWriter * change default mode * investigating parse bug (Garnet SPOP?) * categorise all simulate-connection-failure tests; start working through failures * more simulate-failure tests * update `aoc` test expectation (fast-fail assert in test connect) * optimize ConfigurationOptions fields * Update CI workflow to include v3 branch for packaging * Fix missing Me() key usage * be explicit about test TFMs * merge the sync and async write implementations into a shared implementation that can transition from sync to async if needed * 3.0 release * counting is hard
v3 is now merged into main (v2 branch for maintenance); the version-offset would now lead to weirdness
* - introduce CycleBufferPool as a concept that encapsulates MemoryPool<byte> with the growth logic - support `-d` in the benchmark * - add ConfigurationOptions support - add [SkipLocalsInit] to RESPite * add ResponseArrayPool (#3104) * add ResponseArrayPool * Lease * IMemoryOwner<byte>? --------- Co-authored-by: ITikhonov <ITikhonov@lanit.ru> * refactor API so that MemoryPool<T> can be used throughout, and shared between lease and cycle-buffers * we don't actually need a default CycleBufferPool<T> impl; * fix typo * avoid `Failed` nomenclature in tests - makes it hard to isolate CI problems --------- Co-authored-by: Ivan Tikhonov <pairbit@gmail.com> Co-authored-by: ITikhonov <ITikhonov@lanit.ru>
Updated release notes for version 2.13.17 with fixes and enhancements.
* move KeyIdle[Async]Tests to a server without replication * stabilize Windows CI * don't run the `OBJECT IDLETIME` tests in WSL * skip sentinel tests on Windows CI
* Merge pull request #4 from pairbit/RedisValueSequence Redis value sequence * fix order * fix * PublicAPI.Unshipped * GetCharCount and GetMaxCharCount * add checking OverflowException * comment SequenceCompareTo * implement StartsWith * fix StartsWith * refact * GetHashCode * add Equals, CompareTo, StartsWith for ByteArray or MemoryManager * add ReadOnlySequenceIterator * refac * - GetHashCode - GetChars/GetCharCount - Simplify - normalize .First.Span * TryParse * mark WriteUnifiedSequence(ROS) as redundant for now (and fix 64-bit length) * CompareTo * add integration test (and fix CI) * polishing * avoid the string alloc in GetHashCode (when possible) --------- Co-authored-by: ITikhonov <ITikhonov@lanit.ru> Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
…ShortBlob storage kind (#3114) * spike better RedisValue parsing logic * Introduce new storage kind for small payloads (<=8 bytes) - avoids array alloc * fix CI * CI: tweak failing test * tidying (support ulong as canonical)
3.0 is GA now
* investigate and fix #3123 * F+F the test setup * nit, nuke a using direction * in debug: make all string -> RedisValue *explicit* (incomplete, needs a fix to the ship-file) * fix shipfiles for DEBUG hack * docs * fix exp link * I knew we had a Write...(RedisKey) method somewhere! * move NO ONE to RedisLiterals
* propose agents guidelines and skills * add command source notes * ensure `SORT_RO` parse correctly; impact: Execute would not apply command-map rename correctly otherwise * Update AGENTS.md
* Throw a clear error when an Execute command contains whitespace
Execute("ACL SETUSER x") passes a whole command line as the single command
token, which gets sent as one unknown command and comes back as an opaque
server error. Since a redis command token never contains internal whitespace,
this is always a caller mistake, so the ExecuteMessage constructor now fails
fast with an ArgumentException-style RedisCommandException that names the
offending command and shows the correct token-per-argument form. Resolves #2689.
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
* test for simple space only
limit to simple space; if people are being *that* creative, that's on them
* Remove test case for 'echo\thello' command
Removed test case for command with tab character.
---------
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
Co-authored-by: Marc Gravell <marc.gravell@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.