Thanks for the detailed point-by-point critique. 
Let me start by addressing the weakest parts of my previous post above -
Firstly, whether algo/HFT trading is even dominant enough to warrant our attention/question it
Based on publicly available data,
HFT trading was atleast 23% of all trades on NSE (25% by volume) in 2015 itself.
- Message Traffic:
- ~83% of all messages.
- Order Management:
- ~87% of all order revisions were made by HFTs.
- ~70% of all order cancellations were made by HFTs.
- ~50% of limit order submissions came from HFTs.
Over the subsequent years,
i could only find numbers that clubbed algo and HFT into a single bucket
which has apparently hit ~60% of the market trades.
Secondly, a clear distinction between the various entities involved
Very well spotted.
Not a very clear distinction. Hence the air-quotes around these 3 terms in my previous post.
Upon further thought,
one way to better express what i had in mind is as follows -
-
“Producer”, “Consumer”→ “value-adding” intermediaries- Investors who bear fundamental risk.
- Traders who provide liquidity and aid price discovery.
-
“middlemen”→ “profiteering” intermediaries- Predatory High Frequency Trading, latency arbitrage, quote stuffing, …
At a given price, a buyer AND a seller both often exist for a trade to successfully conclude
i.e. a willing trade occurs with both the participants in the trade valuing themselves to be better-off as perfect information is unlikely in real markets.
Searching online, found a few metrics to objectively identify such profiteering middlemen.
Logical objective metrics that can be used to distinguish between value-adding intermediaries (who provide liquidity and price discovery) and profiteering middle-men (associated with rent extraction, latency arbitrage, quote stuffing, …).
1. Spread Decomposition (Realized vs. Effective Spread)
The Logic:
A benevolent market maker earns the spread but often loses on the price move (adverse selection). A predatory trader profits from the price move itself or by “picking off” stale quotes. This metric compares the cost to the aggressive trader against the profit of the passive provider.
- Metric: Difference between Effective Spread (immediate cost) and Realized Spread (profit after time $t$).
- Distinction:
- Value Adder: High Effective Spread, Low/Negative Price Impact (Realized Spread < Effective Spread). They bear the risk of holding the asset.
- Profiteer: Low Realized Spread for the counter-party or High Price Impact. If the price moves against the victim immediately after the trade, the middle-man likely engaged in latency arbitrage.
2. Order-to-Trade Ratio (OTR) & Cancellation Rates
The Logic:
Value-adding market makers must post quotes to provide liquidity. However, statistically anomalous ratios (e.g., 1000:1) suggest strategies based on the illusion of liquidity rather than provision.
- Metric: Ratio of (New Orders + Modifications + Cancellations) to Executed Trades.
- Distinction:
- Value Adder: Reasonable ratios consistent with market volatility updates.
- Profiteer: Extremely high OTRs indicate Quote Stuffing or Layering—creating “ghost liquidity” that vanishes when accessed, often used to slow down competitors or trigger algorithms.
References: ESMA Final Report on Draft Regulatory Technical Standards (MiFID II)* - Section 3.4 on OTR and
MiFID II RTS 9 Article 2.
3. Inventory Mean Reversion (Holding Period)
The Logic:
Profiteering middle-men (pure arbitrageurs) avoid inventory risk entirely, seeking to end the day (or minute) flat. Investors and genuine Market Makers hold risk for longer periods.
- Metric: Inventory Half-Life. How long does it take for a participant’s inventory to revert to zero?
- Distinction:
- Value Adder: Willing to hold inventory for minutes/hours to facilitate large block trades.
- Profiteer: Inventory reverts to zero in seconds/milliseconds. They effectively transfer risk rather than absorb it.
4. “Strategic Runs” Methodology (Pattern Recognition)
The Logic:
Algorithmic intermediaries often trade in bursts or sequences of trades in the same direction, followed by a reversal. This methodology identifies HFT activity without needing proprietary User IDs.
- Metric: Identification of “runs”—sequences where a participant aggressively buys (or sells) repeatedly within a short window.
- Distinction:
- Momentum Ignition: A predatory strategy where the middle-man trades aggressively to trigger stop-losses.
- Differentiation: Used to separate HFT activity (characterized by speed and inventory management) from low-frequency institutional re-balancing.
Reference: Identifying High Frequency Trading Activity Without Proprietary Data* (Chakrabarty, Comerton-Forde, Pascual).
5. Liquidity Withdrawal During Stress (The “Fair Weather” Test)
The Logic:
A true Market Maker adds value by stabilizing the market during stress. A profiteer provides liquidity only when the probability of loss is near zero and vanishes when volatility spikes.
- Metric: Correlation between Market Volatility (VIX) and Limit Order Book Depth provided by the participant.
- Distinction:
- Value Adder: Low or positive correlation (remains present during stress).
- Profiteer: Strong negative correlation (vanishes exactly when liquidity is needed).
6. Latency Arbitrage Detection
The Logic:
In fragmented markets, prices may misalign for microseconds. Participants identifying and executing against these stale prices add no price discovery value; they tax the system.
- Metric: Identification of trades occurring simultaneously (within light-travel time) across venues where the price difference is mechanical/arbitrage-based.
- Distinction:
- Profiteer: Consistently engages in “sniping” stale quotes.
- Value: Zero. This is widely considered a tax on liquidity.
7. Toxic Flow Analysis (VPIN)
The Logic:
Measures the imbalance of buy/sell volume to detect “toxic” flow—informed traders taking advantage of market makers.
- Metric: VPIN (Volume-Synchronized Probability of Informed Trading).
- Distinction:
- Profiteer: A participant consistently on the aggressive side of high-VPIN periods (utilizing information asymmetry or speed to pick off passive orders).
8. Risk-Adjusted rate of return & Consistency Analysis
The Logic:
Use the Sharpe Ratio (Return / Volatility) and Correlation with Slippage.
Efficient markets dictate that high returns require high risk. HFTs with “infinite” Sharpe ratios are likely extracting risk-free rents (arbitrage/front-running).
- Metric:
- Sharpe Ratio:
(R_p - R_f) / sigma_pcalculated on daily/intraday P&L. - Slippage Correlation: Correlation between HFT Profit and immediate post-trade price movement against the counter-party.
- Sharpe Ratio:
- Distinction:
- Value Adder: Moderate Sharpe Ratio (takes inventory losses). No correlation with immediate adverse price moves (passive liquidity provision).
- Profiteer: Extremely High Sharpe Ratio (20+). High correlation with immediate adverse price moves (sniping/latency arb).
Reference: High-Frequency Trading and the New Market Makers (Albert J Menkveld)
IIUC, access to the historical order-book / bid-ask spreads (or even a dump of live market data eg. broker API web-socket stream) should be sufficient to apply several of these methodologies.
It remains to be seen whether such “profiteering” without value-adding behaviour extends beyond HFT and Algo trading as well.
Also, this updated classification, appears to be consistent with the observation that margins have declined for some value-adding traders, as it doesn’t rule out the presence of other non-value-adding (profiteering) middlemen participants.