Skip to content

[gateio] websocket orders(place,amend,cancel) - #5363

Open
rizer1980 wants to merge 32 commits into
knowm:developfrom
rizer1980:PR/gateio/ws-orders(place,amend,cancel)
Open

rizer1980 wants to merge 32 commits into
knowm:developfrom
rizer1980:PR/gateio/ws-orders(place,amend,cancel)

Conversation

@rizer1980

Copy link
Copy Markdown
Contributor

No description provided.

# Conflicts:
#	xchange-stream-gateio/src/test/java/info/bitrich/xchangestream/gateio/examples/GateioFuturesManualExample.java
#	xchange-stream-gateio/src/test/java/info/bitrich/xchangestream/gateio/examples/GateioSpotManualExample.java

@timmolter timmolter left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this, @rizer1980! WebSocket order placement/amend/cancel for Gate.io is a really nice addition, and the REST-side fixes (reduce-only for EXIT_* types, the t- prefix handling, filled_amount) look like good improvements.

I left a few inline notes on the things that seem most important to sort out before merging, mainly around how order responses are matched and what happens on reconnect, since those could affect real orders. Happy to discuss any of them if I've misread something.

A short PR description would also help other reviewers. Thanks again!

}

@Override
public String getSubscriptionUniqueId(String channelName, Object... args) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to watch out for with modelling each order as a subscription: NettyStreamingService.resubscribeChannels() re-sends getSubscribeMessage for every entry still in channels after a reconnect. If the socket drops while an order is in flight (or a response never arrives, so the entry is never removed), I think the order would be sent again on reconnect, which could mean a duplicate order.

Maybe override resubscribeChannels() here to skip order request ids, or track in-flight requests separately from channels? (Minor related note: the inherited getUnsubscribeMessage returns "", so an empty frame gets sent after each order completes.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hadn't really considered this scenario before. I can't figure out how to check for stuck subscriptions in this case. As a quick fix, I disabled re-subscription for these situations. I also updated the other exchanges where I had implemented similar methods.

payload = GateioWsOrderPayload.<GateioSpotOrderRequest>builder().reqId(reqId).reqParam(reqParam).build();
break;
}
case Config.FUTURES_ORDER_CANCEL_CHANNEL: {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cancelOrder uses SPOT_ORDER_CANCEL_CHANNEL for spot instruments, but there's no case for it here, so I think spot cancels fall through to EmptyPayload. Also, currency_pair is set from instrument.toString() (e.g. BTC/USDT), so it might need GateioAdapters.toGateioInstrument(...) to get BTC_USDT.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

}
String size = null;
if (limitOrder.getOriginalAmount() != null) {
size = limitOrder.getType() == Order.OrderType.BID | limitOrder.getType() == Order.OrderType.EXIT_ASK

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like changeOrder passes contractValue as args[2], but it isn't used here, so size goes out in base units rather than contracts. Placement converts via convertVolumeToContractSize. With a contract value of 0.01, I think amending to 1 BTC would end up as 1 contract (0.01 BTC). Could we apply the same conversion here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my mistake, fixed

text(limitOrder.getUserReference() != null ? limitOrder.getUserReference() : null).
build();
return builder.contract(toGateioInstrument(limitOrder.getInstrument()))
.size(limitOrder.getType() == OrderType.BID ? size.toPlainString() : size.negate().toPlainString())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small one: the market-order version was updated to treat EXIT_ASK as a buy (BID | EXIT_ASK), but this limit-order version still only checks BID. So I think an EXIT_ASK limit order (closing a short) would be sent with a negative size. Also, this path doesn't go through formatUserReference, so text may be missing the t- prefix Gate requires.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.
Thanks for such a detailed breakdown of the errors.

@timmolter timmolter left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround on the first round, @rizer1980. The ack handling, the resubscribe guard, the spot cancel channel and the amend contract conversion all look fixed. I pulled the branch and the tests in xchange-gateio-v4 and xchange-stream-gateio pass.

A few new things came up on the second pass, mostly in how the fixes interact with the connection lifecycle. The ones I'd consider blocking are the 1-second timeout returning -1 (possible duplicate orders), the spot TypeReference copy-paste, the originalAmount regression for limit buys, and the missing t- prefix on futures limit orders. The rest are smaller.

One more: the branch is still on 6.0.0-SNAPSHOT. develop is now 6.0.1-SNAPSHOT after the 6.0.0 release, so it needs a rebase before merge.

return label != null ? GateioErrorLabels.convert(label) : -1;
}
})
.timeout(1, TimeUnit.SECONDS, Observable.just(-1))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the ack handling, that looks right now. Two things on the timeout though:

  1. One second between the ack and the final result feels tight for a live exchange, and a timeout here yields -1, which is the same value a real rejection produces. A caller that sees -1 would reasonably retry, and then the original order fills a moment later, so we'd end up with a duplicate live order. Could the timeout be longer (several seconds) and surface as an error (Single.error(new TimeoutException(...))) rather than -1, so callers can tell "unknown outcome" apart from "rejected"?
  2. The ack itself (shared.take(1) before the flatMap) has no timeout, so if Gate never answers, or answers without a request_id (getChannelNameFromMessage returns "" in that case), the Single never completes. Applying one .timeout(...) to the whole observable before firstElement() would cover both.

Same applies to the other three copies of this block.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked the delays to make them adjustable(WS_CONNECTION_TIMEOUT). Hope I've fixed the "no timeout" issues.

} else {
Observable<Integer> observable = userTradeStreamingService.subscribeChannel(SPOT_ORDER_PLACE_CHANNEL, reqId, marketOrder)
.map(node -> {
TypeReference<GateioUserTradeWsResponse<GateioWsOrderPayload<GateioFuturesOrderRequest>>> typeReference =

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spot branch still deserializes with GateioWsOrderPayload<GateioFuturesOrderRequest> (looks like a copy-paste from the futures branch above). It should be GateioSpotOrderRequest, as in placeLimitOrder. It only works today because FAIL_ON_UNKNOWN_PROPERTIES is off, so the req_param content is silently dropped.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed


@Override
public Single<Integer> placeLimitOrder(LimitOrder limitOrder, Object... args) {
String reqId = System.currentTimeMillis() + String.valueOf(System.nanoTime() / 1000 % 1000);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: this id is millis + (nanoTime/1000 % 1000), repeated in each of the five methods. Two orders in the same millisecond can collide, and NettyStreamingService.subscribeChannel uses computeIfAbsent, so a colliding second order would silently reuse the first subscription and never be sent. IdGenerator.getInstance().requestId() already exists and is used for id in getWsRequest, so maybe use it here too.

Related: the ack/result publish(...) block is now duplicated four times. Pulling it into one sendOrderRequest(channel, typeRef, args...) helper would mean fixes like the timeout and the TypeReference above only need to land once. The unused GateioWsPlaceOrderSpotResponse / GateioWsPlaceOrderFutureResponse DTOs and the commented-out initServices() / builder.account("unified") blocks could probably go as well.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System.nanoTime returns nanoseconds; dividing by 1,000 gives us microseconds, which is 1,000 times more precise than milliseconds. result is System.currentTimeMillis() long and microseconds in one string. So, collisions are unlikely. For some reason, I find it convenient to use a timestamp rather than a random ID. This could be useful for analyzing request processing time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove commented code. optimize placeLimit and placeMarket websockets orders. remove unused class

return builder.contract(toGateioInstrument(limitOrder.getInstrument()))
.size(getGateioSize(limitOrder, size))
.price(limitOrder.getLimitPrice().toPlainString())
.text(limitOrder.getUserReference() != null ? limitOrder.getUserReference() : null)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The t- prefix fix landed for spot and for the futures market order, but this futures limit order path still passes the raw user reference as text. Could we run it through formatUserReference(...) here too? Otherwise a futures limit order with a user reference is rejected while the same reference on a market order succeeds.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

// .getFilledTotalQuote()
// .divide(gateioOrder.getAvgDealPrice(), MathContext.DECIMAL32);
builder.cumulativeAmount(gateioOrder.getFilledAmount())
.originalAmount(null);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this changes behaviour for limit buys. Before this PR originalAmount was always set from amount; now every filled or partially filled BID gets null, including type == "limit" where amount is already the base quantity. Callers computing remaining = original - cumulative would NPE, and it also flows into the streaming spot.orders updates via GateioStreamingAdapters. Could the null be limited to "market".equals(gateioOrder.getType()), since that's the only case where amount is quote-denominated?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

OrderQueryParamInstrument params = (OrderQueryParamInstrument) orderQueryParams[0];

String orderId = "";
Instrument instrument = null;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small one: the Validate.isInstanceOf(OrderQueryParamInstrument.class, ...) guard was dropped, so a plain DefaultQueryOrderParam now reaches the spot getOrder(orderId, null) with currency_pair=null instead of failing with a clear IllegalStateException. If the intent is to support instrument-less params, fine, but then the null instrument probably needs handling; otherwise I'd keep the guard.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currency_pair=null it’s not being passed to the request. I added a test and also verified it manually — everything is fine.

userTradeStreamingService =
new GateioUserTradeStreamingService(exchangeSpecification.getSslUri(), exchangeSpecification.getApiKey(),
exchangeSpecification.getSecretKey(), exchangeSpecification);
new GateioUserTradeStreamingService(exchangeSpecification.getSslUri(),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

disconnect() below only tears down streamingService. userTradeStreamingService is never disconnected, so after exchange.disconnect() the authenticated socket stays open and pingPongSubscription keeps sending pings every 15s. Repeated connect/disconnect cycles leak sockets and timers. Could disconnect() also call userTradeStreamingService.disconnect() and null it out?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed


@Override
public Completable disconnect() {
compositeDisposable.dispose();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compositeDisposable.dispose() is terminal: after a connect() -> disconnect() -> connect() cycle, the subscribeDisconnect() subscription added in the second connect() is disposed immediately, so loginDone is never reset on a later network drop and isAlive() keeps returning true on an unauthenticated reconnected socket. compositeDisposable.clear() would fix that. It would also be good to dispose pingPongSubscription here. The same dispose() pattern is in BybitUserTradeStreamingService from this PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, not sure by 100%, that is correct now.
pingPongSubscription is managed in GateioStreamingExchange.disconnect

loginDone = false;
});
compositeDisposable.add(disposable);
completable.onComplete();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

connect() completes as soon as the login request is sent, before the login ack has set loginDone. A caller doing connect().blockingAwait() followed immediately by placeMarketOrder(...) would send order_place on a not-yet-authenticated socket and get -1. Also, a non-200 login response is swallowed in messageHandler (it just returns), so with a bad API key connect() reports success and every order silently returns -1.

Could we complete the Completable only once the login ack arrives (and onError on a non-200 status), with a timeout?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

.channel(channelName)
.event(Event.API.getValue())
.time(Instant.now(Config.getInstance().getClock()))
.time(Config.getInstance().getClock().millis())

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

time here is milliseconds, while the public GateioWsRequest serializes time in seconds and Gate's WS API docs describe time as Unix seconds. Login uses millis for time but seconds for timestamp inside the payload. Since Gate validates request time against server time for API-event requests, it's probably safer to send seconds consistently.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backward compatibility issues: the gate io used to work with seconds, but later migrated to milliseconds.
I’ve added a field for milliseconds, time_ms; I believe everything is correct now.

(setDualMode, setPositionMode)
# Conflicts:
#	xchange-stream-binance/src/main/java/info/bitrich/xchangestream/binance/BinanceUserDataSpotStreamingService.java
#	xchange-stream-binance/src/main/java/info/bitrich/xchangestream/binance/BinanceUserTradeStreamingService.java
…everage-setting method on Gate.io: a single JSON object (one-way / single mode) and an array of objects (dual / hedge mode).
…e once authentication is confirmed via the login acknowledgment response.

- Handled non-200 login responses by failing the `Completable` with an `ExchangeSecurityException` and added connection timeout handling.
… but later migrated to milliseconds.

I’ve added a field for milliseconds, `time_ms`; I believe everything is correct now.
@rizer1980

Copy link
Copy Markdown
Contributor Author

i think, it's ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants