2025-10-10 14:07:03,040 [42] ERROR MCMS.Portal.Common.GlobalExceptionFilter - URL:http://portal.10g10g.com/UserCommonLocations/GetLocationByIP?ip=::1;UserIP:127.0.0.1,ServerVariables:Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestHeaders;Exception:Input string was not in a correct format. at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type) at System.Int64.Parse(String s) at MCMS.Portal.DAO.Hibernate.IpLocationDAO.<>c.b__3_0(String s) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at System.Linq.Enumerable.SelectArrayIterator`2.ToList() at MCMS.Portal.DAO.Hibernate.IpLocationDAO.IpToInt(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at MCMS.Portal.DAO.Hibernate.IpLocationDAO.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 26 at MCMS.Portal.Service.Impl.IpLocationService.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.Service.Impl\IpLocationService.cs:line 131 at _dynamic_MCMS.Portal.Service.Impl.IpLocationService.GetByIp(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_5639c95a424c408387b61ea027b16411.GetByIp(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(String ip) at _dynamic_MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_53ef54187ee54ce398832382e2df47d8.GetLocationByIP(String ip) at MCMS.Portal.Controllers.UserCommonLocationsController.GetLocationByIP(String ip) at lambda_method399(Closure , Object , Object[] ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2025-10-10 14:07:03,337 [199] ERROR MCMS.Portal.Common.GlobalExceptionFilter - URL:http://portal.10g10g.com/UserCommonLocations/IsCommonLocations?userId=1006&ip=::1;UserIP:127.0.0.1,ServerVariables:Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestHeaders;Exception:Input string was not in a correct format. at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type) at System.Int64.Parse(String s) at MCMS.Portal.DAO.Hibernate.IpLocationDAO.<>c.b__3_0(String s) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at System.Linq.Enumerable.SelectArrayIterator`2.ToList() at MCMS.Portal.DAO.Hibernate.IpLocationDAO.IpToInt(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at MCMS.Portal.DAO.Hibernate.IpLocationDAO.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 26 at MCMS.Portal.Service.Impl.IpLocationService.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.Service.Impl\IpLocationService.cs:line 131 at _dynamic_MCMS.Portal.Service.Impl.IpLocationService.GetByIp(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_5639c95a424c408387b61ea027b16411.GetByIp(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.IsCommonLocations(Int32 userId, String ip) at _dynamic_MCMS.Portal.Service.Impl.UserCommonLocationsService.IsCommonLocations(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_53ef54187ee54ce398832382e2df47d8.IsCommonLocations(Int32 userId, String ip) at MCMS.Portal.Controllers.UserCommonLocationsController.IsCommonLocations(Int32 userId, String ip) at lambda_method408(Closure , Object , Object[] ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2025-10-10 14:34:01,292 [91] ERROR MCMS.Portal.Common.GlobalExceptionFilter - URL:http://portal.10g10g.com/UserCommonLocations/GetLocationByIP?ip=::1;UserIP:127.0.0.1,ServerVariables:Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestHeaders;Exception:Input string was not in a correct format. at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type) at System.Int64.Parse(String s) at MCMS.Portal.DAO.Hibernate.IpLocationDAO.<>c.b__3_0(String s) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at System.Linq.Enumerable.SelectArrayIterator`2.ToList() at MCMS.Portal.DAO.Hibernate.IpLocationDAO.IpToInt(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at MCMS.Portal.DAO.Hibernate.IpLocationDAO.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 26 at MCMS.Portal.Service.Impl.IpLocationService.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.Service.Impl\IpLocationService.cs:line 131 at _dynamic_MCMS.Portal.Service.Impl.IpLocationService.GetByIp(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_5639c95a424c408387b61ea027b16411.GetByIp(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(String ip) at _dynamic_MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_53ef54187ee54ce398832382e2df47d8.GetLocationByIP(String ip) at MCMS.Portal.Controllers.UserCommonLocationsController.GetLocationByIP(String ip) at lambda_method399(Closure , Object , Object[] ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2025-10-10 14:34:01,734 [84] ERROR MCMS.Portal.Common.GlobalExceptionFilter - URL:http://portal.10g10g.com/UserCommonLocations/IsCommonLocations?userId=1&ip=::1;UserIP:127.0.0.1,ServerVariables:Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestHeaders;Exception:Input string was not in a correct format. at System.Number.ThrowOverflowOrFormatException(ParsingStatus status, TypeCode type) at System.Int64.Parse(String s) at MCMS.Portal.DAO.Hibernate.IpLocationDAO.<>c.b__3_0(String s) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at System.Linq.Enumerable.SelectArrayIterator`2.ToList() at MCMS.Portal.DAO.Hibernate.IpLocationDAO.IpToInt(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 83 at MCMS.Portal.DAO.Hibernate.IpLocationDAO.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.DAO.Hibernate\IpLocationDAO.cs:line 26 at MCMS.Portal.Service.Impl.IpLocationService.GetByIp(String ip) in E:\agent\_work\8\s\MCMS.Portal.Service.Impl\IpLocationService.cs:line 131 at _dynamic_MCMS.Portal.Service.Impl.IpLocationService.GetByIp(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_5639c95a424c408387b61ea027b16411.GetByIp(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.GetLocationByIP(String ip) at MCMS.Portal.Service.Impl.UserCommonLocationsService.IsCommonLocations(Int32 userId, String ip) at _dynamic_MCMS.Portal.Service.Impl.UserCommonLocationsService.IsCommonLocations(Object , Object[] ) at Spring.Reflection.Dynamic.SafeMethod.Invoke(Object target, Object[] arguments) at Spring.Aop.Framework.DynamicMethodInvocation.InvokeJoinpoint() at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Transaction.Interceptor.TransactionInterceptor.Invoke(IMethodInvocation invocation) at Spring.Aop.Framework.AbstractMethodInvocation.Proceed() at Spring.Aop.Framework.DynamicProxy.AdvisedProxy.Invoke(Object proxy, Object target, Type targetType, MethodInfo targetMethod, MethodInfo proxyMethod, Object[] args, IList interceptors) at CompositionAopProxy_53ef54187ee54ce398832382e2df47d8.IsCommonLocations(Int32 userId, String ip) at MCMS.Portal.Controllers.UserCommonLocationsController.IsCommonLocations(Int32 userId, String ip) at lambda_method408(Closure , Object , Object[] ) at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() --- End of stack trace from previous location --- at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|26_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) 2025-10-10 14:57:58,994 [307] ERROR DotNetCore.CAP.RedisStreams.RedisStreamManager - Redis error while creating consumer group [cap.queue.mcms.portal.v1] of stream [MCMS.Portal.SetCustomerFirstOrderUserId] StackExchange.Redis.RedisTimeoutException: Timeout awaiting response (outbound=0KiB, inbound=0KiB, 3376ms elapsed, timeout is 3000ms), command=EXISTS, next: EXISTS MCMS.Portal.SetCustomerFirstOrderUserId, inst: 0, qu: 0, qs: 1, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, in-pipe: 0, out-pipe: 0, last-in: 0, cur-in: 0, sync-ops: 0, async-ops: 76437174, serverEndpoint: 127.0.0.1:16379, conn-sec: 3367275.08, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: DevelopmentServer(SE.Redis-v2.8.58.19568), IOCP: (Busy=0,Free=1000,Min=100,Max=1000), WORKER: (Busy=1,Free=32766,Min=100,Max=32767), POOL: (Threads=8,QueuedItems=0,CompletedItems=245419177,Timers=10), v: 2.8.58.19568 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateStreamConsumerGroupAsync(IDatabase database, RedisKey stream, RedisValue consumerGroup) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateConsumerGroupPositionsAsync(IDatabase database, StreamPosition[] positions, String consumerGroup, ILogger logger)+MoveNext() 2025-10-10 14:59:38,092 [41] ERROR DotNetCore.CAP.RedisStreams.RedisStreamManager - Redis error while creating consumer group [cap.queue.mcms.portal.v1] of stream [MCMS.Portal.SetCustomerFirstOrderUserId] StackExchange.Redis.RedisTimeoutException: Timeout awaiting response (outbound=0KiB, inbound=0KiB, 4525ms elapsed, timeout is 3000ms), command=EXISTS, next: EXISTS MCMS.Portal.SetCustomerFirstOrderUserId, inst: 0, qu: 0, qs: 1, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, in-pipe: 0, out-pipe: 0, last-in: 0, cur-in: 0, sync-ops: 0, async-ops: 76437196, serverEndpoint: 127.0.0.1:16379, conn-sec: 3367287.06, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: DevelopmentServer(SE.Redis-v2.8.58.19568), IOCP: (Busy=0,Free=1000,Min=100,Max=1000), WORKER: (Busy=4,Free=32763,Min=100,Max=32767), POOL: (Threads=9,QueuedItems=1,CompletedItems=245419276,Timers=10), v: 2.8.58.19568 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateStreamConsumerGroupAsync(IDatabase database, RedisKey stream, RedisValue consumerGroup) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateConsumerGroupPositionsAsync(IDatabase database, StreamPosition[] positions, String consumerGroup, ILogger logger)+MoveNext() 2025-10-10 14:59:38,192 [92] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.TryResetConnectionAsync(ConnectionSettings cs, MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 650 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 14:59:38,192 [263] WARN DotNetCore.CAP.Processor.MessageDelayedProcessor - Get delayed messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.TryResetConnectionAsync(ConnectionSettings cs, MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 650 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.MySqlDataStorage.ScheduleMessagesOfDelayedAsync(Func`3 scheduleTask, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.ScheduleMessagesOfDelayedAsync(Func`3 scheduleTask, CancellationToken token) at DotNetCore.CAP.Processor.MessageDelayedProcessor.ProcessDelayedAsync(IDataStorage connection, ProcessingContext context) 2025-10-10 14:59:38,394 [283] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 14:59:38,394 [328] ERROR DotNetCore.CAP.Processor.CollectorProcessor - An error occurred while attempting to delete expired data from table 'cap.published':Connect Timeout expired. MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteNonQueryAsync(DbConnection connection, String sql, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.Processor.CollectorProcessor.ProcessAsync(ProcessingContext context) 2025-10-10 14:59:38,407 [328] WARN DotNetCore.CAP.Processor.InfiniteRetryProcessor - Processor 'DotNetCore.CAP.Processor.CollectorProcessor' failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteNonQueryAsync(DbConnection connection, String sql, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.Processor.CollectorProcessor.ProcessAsync(ProcessingContext context) at DotNetCore.CAP.Processor.InfiniteRetryProcessor.ProcessAsync(ProcessingContext context) 2025-10-10 15:05:09,787 [177] ERROR DotNetCore.CAP.RedisStreams.RedisStreamManager - Redis error while creating consumer group [cap.queue.mcms.portal.v1] of stream [MCMS.Portal.ClearCustomerFirstOrderUserId] StackExchange.Redis.RedisTimeoutException: Timeout awaiting response (outbound=0KiB, inbound=0KiB, 5792ms elapsed, timeout is 3000ms), command=XINFO, next: XINFO MCMS.Portal.ClearCustomerFirstOrderUserId, inst: 0, qu: 0, qs: 1, aw: False, bw: SpinningDown, rs: ReadAsync, ws: Idle, in: 0, in-pipe: 0, out-pipe: 0, last-in: 1, cur-in: 0, sync-ops: 0, async-ops: 76437198, serverEndpoint: 127.0.0.1:16379, conn-sec: 3367394.14, aoc: 0, mc: 1/1/0, mgr: 10 of 10 available, clientName: DevelopmentServer(SE.Redis-v2.8.58.19568), IOCP: (Busy=0,Free=1000,Min=100,Max=1000), WORKER: (Busy=2,Free=32765,Min=100,Max=32767), POOL: (Threads=13,QueuedItems=0,CompletedItems=245419379,Timers=11), v: 2.8.58.19568 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateStreamGroupAsync(IDatabase database, RedisKey stream, RedisValue consumerGroup) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateStreamGroupAsync(IDatabase database, RedisKey stream, RedisValue consumerGroup) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateStreamConsumerGroupAsync(IDatabase database, RedisKey stream, RedisValue consumerGroup) at DotNetCore.CAP.RedisStreams.RedisStreamManagerExtensions.TryGetOrCreateConsumerGroupPositionsAsync(IDatabase database, StreamPosition[] positions, String consumerGroup, ILogger logger)+MoveNext() 2025-10-10 15:05:09,795 [318] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,797 [328] WARN DotNetCore.CAP.Processor.MessageDelayedProcessor - Get delayed messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.MySqlDataStorage.ScheduleMessagesOfDelayedAsync(Func`3 scheduleTask, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.ScheduleMessagesOfDelayedAsync(Func`3 scheduleTask, CancellationToken token) at DotNetCore.CAP.Processor.MessageDelayedProcessor.ProcessDelayedAsync(IDataStorage connection, ProcessingContext context) 2025-10-10 15:05:09,798 [33] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,799 [359] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,800 [254] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,801 [188] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,802 [82] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,806 [271] ERROR DotNetCore.CAP.Processor.CollectorProcessor - An error occurred while attempting to delete expired data from table 'cap.published':Connect Timeout expired. MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteNonQueryAsync(DbConnection connection, String sql, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.Processor.CollectorProcessor.ProcessAsync(ProcessingContext context) 2025-10-10 15:05:09,807 [175] WARN DotNetCore.CAP.Processor.MessageNeedToRetryProcessor - Get messages from storage failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteReaderAsync[T](DbConnection connection, String sql, Func`2 readerFunc, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.MySql.MySqlDataStorage.GetMessagesOfNeedRetryAsync(String tableName, TimeSpan lookbackSeconds) at DotNetCore.CAP.Processor.MessageNeedToRetryProcessor.GetSafelyAsync[T](Func`2 getMessagesAsync, TimeSpan lookbackSeconds) 2025-10-10 15:05:09,846 [Kestrel Timer] WARN Microsoft.AspNetCore.Server.Kestrel - As of "10/10/2025 07:00:05 +00:00", the heartbeat has been running for "00:00:01.4376350" which is longer than "00:00:01". This could be caused by thread pool starvation. 2025-10-10 15:05:09,867 [271] WARN DotNetCore.CAP.Processor.InfiniteRetryProcessor - Processor 'DotNetCore.CAP.Processor.CollectorProcessor' failed. Retrying... MySqlConnector.MySqlException (0x80004005): Connect Timeout expired. ---> MySqlConnector.MySqlException (0x80004005): The Command Timeout expired before the operation completed. ---> System.Net.Sockets.SocketException (125): Operation canceled at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.SocketByteHandler.DoReadBytesAsync(Memory`1 buffer) in /_/src/MySqlConnector/Protocol/Serialization/SocketByteHandler.cs:line 109 at MySqlConnector.Protocol.Serialization.BufferedByteReader.ReadBytesAsync(IByteHandler byteHandler, ArraySegment`1 buffer, Int32 totalBytesToRead, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/BufferedByteReader.cs:line 34 at MySqlConnector.Protocol.Serialization.ProtocolUtility.ReadPayloadAsync(BufferedByteReader bufferedByteReader, IByteHandler byteHandler, Func`1 getNextSequenceNumber, ArraySegmentHolder`1 previousPayloads, ProtocolErrorBehavior protocolErrorBehavior, IOBehavior ioBehavior) in /_/src/MySqlConnector/Protocol/Serialization/ProtocolUtility.cs:line 421 at MySqlConnector.Core.ServerSession.ReceiveReplyAsync(IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 886 at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 442 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427 at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432 at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405 at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 144 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1103 at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 590 at DotNetCore.CAP.MySql.DbConnectionExtensions.ExecuteNonQueryAsync(DbConnection connection, String sql, DbTransaction transaction, Object[] sqlParams) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.MySql.MySqlDataStorage.DeleteExpiresAsync(String table, DateTime timeout, Int32 batchCount, CancellationToken token) at DotNetCore.CAP.Processor.CollectorProcessor.ProcessAsync(ProcessingContext context) at DotNetCore.CAP.Processor.InfiniteRetryProcessor.ProcessAsync(ProcessingContext context) 2025-10-10 15:05:09,927 [33] WARN DotNetCore.CAP.RedisStreams.AsyncLazyRedisConnection - Connection restored back!, (null), for endpoint:127.0.0.1:16379, failure type:None, connection type:Interactive 2025-10-10 15:29:32,675 [358] DEBUG CapSubscribeService - SubscribeCustomerAddGrandTotal接收到订阅:{"Id":128343,"Money":1.00,"Note":"订单【202510101426478611】发货"} 2025-10-10 15:31:44,110 [318] DEBUG CapSubscribeService - SubscribeUpdateCustomerPrimaryLinkWechatInfo接收到订阅:{"CustomerNumber":"29B162DEFB95DD8F","OrderNumber":"202510101426478611","OrderUserId":268,"OpUser":"系统管理员","OpNote":"订单(202510101426478611)已完成"} 2025-10-10 15:31:45,465 [139] DEBUG CapSubscribeService - SubscribeUpdateLastSoldTime接收到订阅:{"CustomerId":128343,"SoldTime":"2025-10-10 14:26:47","UserId":268,"UserSoldTime":"2025-10-10 14:26:47"} 2025-10-10 15:31:45,470 [139] DEBUG CustomerInfoService - 客户编号:29B162DEFB95DD8F