This is on held-out data. These weighting patterns fascinate me! Everything is so cyclical, but just look at how it flattens out the dip, and how profitable it is on the second day without missing a beat! Sure the actual weights need revision but the pattern!

The pattern is not common to all days, just these 2 particular days. Are weird cyclical weighting patterns like this common?

In a previous post that I made here (and in r/ValueInvesting) earlier this year, I presented the results of a machine learning model that used fundamentals from stocks listed on the S&P 100 to predict stock returns and then used quarterly re-balancing. While I thought that this method worked well, I have had problems with my data provider (SimFin) and noticed that tickers were not consistently present in my dataset, in addition to some errors.

As such, I abandoned that approach and decided to directly pull the data from SEC's EDGAR database. In addition, I wanted to test out a rules-based approach that did not use machine learning and required even less turnover/maintenance.

To that end, I would like to present my current system which uses 9 factors (8 fundamental factors and 1 technical indicator) to pick a basket of stocks at the end of each year and hold them for the entirety of the following year.

The Universe: I focused on S&P 100 stocks for several reasons. For one, I was able to build a fairly robust constituent list going back a decade using the Wayback Machine and Wikipedia; I essentially pulled the constituent list as close to the end of the year as I could and use that for the following year. Another reason is that it is challenging to get stock data for tickers that no longer exist - this accounts for dozens of tickers in the S&P 500, including numerous bankruptcies. The reason a ticker may not be included in the SP100 universe is usually due to acquisitions, rather than bankruptcies. As such I believe my back test is likely more accurate for the SP100 rather than SP500 universe. That being said, the number of tickers in my backtest grows over the years from 54 in 2016 to 82 in 2025. This is a real caveat as well and due to numerous tickers for companies that were acquired and thus no longer have stock data on yahoo finance. This could cut both ways in the sense that we miss the bump that M&A may lead to but also miss the losses associated with poor performers that may have been ranked high in their fundamentals.

The Data: I pulled data from SEC's EDGAR system and focused on pulling 10-Qs AND 10-Ks published during the last calendar quarter (October 1 - December 31) regardless of the company's own fiscal calendar. In cases where the 10K wasn't available, 10Qs was used to produce the same metrics. Only companies with complete data for all factors were included.

The Factors: I used the following 9 factors (8 fundamentals and 1 technical indicator) to rank stocks. The factor and the category they fall into are:

• Profitability & Efficiency: Return on Assets (TTM), Operating Margin (TTM).
• Cash Flow Quality & Accruals: Free Cash Flow to Assets, Cash Flow to Net Income Ratio.
• Solvency & Capital Structure: Current Ratio, Share Dilution (inverted).
• Growth Dynamics: YoY Revenue Growth, YoY Net Income Growth.
• Price Momentum: 1-Year Trailing Return.

Each factor is converted into a standardized score; all scores are then averaged to create a final score/ranking.

The 8 fundamental factors were selected a priori, and I added momentum later after the first batch of testing just based on a hunch. Beyond that, the factors were not further iterated upon.

Trade Execution: Trades are quite simple; we simply buy x top scored stocks on the first trading day of the year and hold until the last trading day of the year.

Results

Total returns from 2016 to present are below for three following basket sizes: 10, 15 and 20.

Yearly results are presented below:

Overall, all three baskets outperform SPY by roughly ~300% to 500% over a 10-year period and produce better CAGRs, Sharpe scores, and drawdowns. On a yearly basis, the top 10 basket outperformed SPY 8/11 years (including YTD); the Top 15 and 20 baskets outperformed SPY 9/11 years.

I conducted further statistical testing using the top 10 basket to determine whether my results were statistically significant.

The results of the Monte Carlo testing (1000 random draws) are below:

As can be seen, our results are above beyond luck (p = 0.003) and a t-test comparing yearly returns (should be taken with a grain of salt given there are only 11 years) are also significant suggesting that our results are not a function of luck.

To further test whether our returns were driven by a tech bull run, I ran a linear regression that sought to determine whether our returns could be predicted by XLK's performance. When running the regression on yearly returns the result is not significant (p = 0.3); however - as with the pairwise t-test, our n is only 11; as such I ran the same regression on monthly returns (which expands our sample size to ~130) and this model was significant (p < 0.001; R2 = 0.2946). This means that 30% of the variance seen in our returns can be explained by XLK. Considering how much tech companies have driven broader market returns and the S&P 100 universe we are selecting from, this is largely unavoidable. Ultimately, this remains a strong result in my opinion as 70% of the variance in our returns is independent of the tech sector bull run.

Overall, I think these results are promising but would love to hear what people think. Happy to answer any questions you may have!

Hey r/algotrading,

pandas-ta-classic v0.9.0 is out. This is the community-maintained fork of pandas-ta. This release is about one theme: the library no longer hides mistakes. Code that got a default or None back now gets an exception, and results that were wrong now change. It is a breaking release, so please read the upgrade notes before you upgrade.

🚨 Breaking: mistakes now raise Invalid arguments raise ValueError instead of being replaced by a default. Non-Series input raises TypeError. Input shorter than the window returns an all-NaN result instead of None.

ta.sma(close, length=0)        # 0.8.x: SMA_10   0.9.0: ValueError
ta.sma(close.values)           # 0.8.x: None     0.9.0: TypeError
ta.sma(close[:10], length=50)  # 0.8.x: None     0.9.0: all-NaN SMA_50

Replace if result is None with if result.isna().all().

🕳️ Missing data and warm-up - One missing bar erased the rest of the series (NaN to the end) for fisher, macd, kama, mama, the ht_* family and others. Now the missing bar reads NaN, and the indicator carries on after it. - 48 indicators published a value on a bar where every input was missing (ema, rsi, atr, adx, psar, supertrend, ...). Now that bar reads NaN. - One NaN row (which resample('D') adds on every weekend) made candle patterns read 0 for the rest of the series: 41 of 62 never fired. Now they keep detecting. - Warm-up bars read NaN instead of placeholders or guesses. Flag columns that can hold NaN are float64: test them with == 1, not astype(bool).

🎯 Correctness and TA-Lib parity - A flat window (no range, movement or variance) reads 0.0, the value TA-Lib uses. Before, it gave NaN for whole columns, +inf, or values up to 3.6e19. - Indicators fixed to match their definitions: vfi read 0.0 on every bar (it now takes high and low), ebsw used degrees as radians, kst was 100× too large, and stc, cksp, trima, supertrend and cpr had smaller bugs. - Performance metrics: total_time counted 252 trading days per calendar year, so a series that doubles in one year reported a CAGR of ~61%. It now reports ~100%. - talib=True honours every parameter. Before, it silently ignored any parameter that TA-Lib cannot express.

🔧 strategy() - df.ta.strategy() runs serially by default. Use cores=8 or the new executor= to run in parallel. - Chained entries such as {"kind": "ema", "close": "SMA_10"} were silently lost when they ran on workers. Now serial and parallel runs give the same columns. - An unknown strategy name, a malformed exclude or a wrong col_names raises an error instead of doing nothing.

⚡ Performance (with numba) | Indicator | Speedup (100,000 bars) | |---|---| | ht_* family | ~110× | | macd / adosc / vidya | 25× / 17× / 8× | | true_range, atr, adx family | 2–8× |

🗑️ Removed - Python 3.10. Supported versions are 3.11–3.15. - Built-in data fetching (df.ta.ticker(), ta.yf(), ta.av()). Use yfinance directly. - df.ta.constants(). - The ichimoku() tuple. It now returns one DataFrame.

📦 Install

pip install -U pandas-ta-classic

224 indicators plus 62 native candlestick patterns, and TA-Lib is not required.

Upgrade notes: https://github.com/xgboosted/pandas-ta-classic/blob/main/CHANGELOG.md#upgrading-from-08x GitHub: https://github.com/xgboosted/pandas-ta-classic

If your backtest numbers change after the upgrade, I want to know. Most changes fix values that were wrong, but please check. Bug reports with a small reproduction help the most.

Community maintained. PRs open.

I asked this a short while ago and was so amazed with the wide variety of things everyone was building, large and small. It was eye opening and motivating to recognize all your genius and effort.

Today I am improving my backtesting system. Once upon a time a single back test was scrolling through a nasty list of ~400 inputs, recording results manually into excel, aligning timelines, organizing settings so I do not confuse configurations with their backtest, and about 5 more tedious steps... Today it all happens automatically in a couple clicks.

Build, build, build, and you will reach your goals. It is inevitable.

What are you building today?

The main idea is simple, just wanted to be able to hook up my polymarket, choose an amount, and copy insiders- buy when they buy, sell when they sell. Unusual Whales is great but charges money, and since the data is free so I wanted to build a baseline and let others play around with it, see if it actually works for them and have total control over it to improve it.

Am I missing something obvious?

Try it first: polymarket-trader-lac.vercel.app. Code repo linked at the top of the demo. Paper money, credentials are pre-filled and its shared sandbox with synthetic data, resets every 2 hours).

My one main issue right now is that paper trading looked good but live trading did not fill as fast, so I held contracts longer than the paper sim assumed and gave back some profit. Fill delay is now modeled in paper mode but again its a rough assumption. Unsure how to resolve this, feedback welcome!

Other issues have to do with how to track and identify "insider" Wallets in the first place.

Disclaimer: Not financial advice.

I have recently been getting into creating and testing models. The original intent was to remove myself from the picture as I believe emotion adds too much uncertainty to be successful over the long run if you are not able to curb it. Unfortunately, after months of testing I have realized that the way I actually trade is incredibly difficult to actually program. The main reason, my strategy adapts. Any algo will follow strict set rules and will not bend. But there are times where I will ignore a signal if other conditions exist, or make a move that has not officially signaled yet if I have high conviction we are going that way. For instance, I may program my rule to exit within 10% of the next pivot point to capture profit and reduce giveback. But managing the play live, I may hit that 10% barrier and take into account that the last 3 candles were all in my direction and growing in size so I may extend to over the next pivot point.

This is just one example. I realize I could program this into the algo, but to be able to do this for every conceivable point of flex that my system would need would require hundreds, possibly thousands of these little rules and some would obviously contradict with each other.

At this point, I have several models that I am carrying into paper trading. None of them really capture how I actually trade. But all of them attempt to capture a simplified version of one aspect of how I trade on any given day. As such, it naturally limits large winners (3-10 baggers) where I can hit them at a decent pace controlling it myself. So at this point, my idea to remove myself completely has adapted. I will now use my algos to do the majority of my trading, then set aside a set amount for manual high risk/return style plays.

But my question remains. How do you guys get around these obstacles when you are making your algos? Do you end up breaking strats into chunks like I am doing? Or have you found a way to actually build in adaptability to your models?

total file size is 34 gb

Now I'm just sitting here with all of it and running out of ideas on what to actually do.

if you guys had this what would you do

So, the code can predict x candles in the future, and from an offset. e.g. predict the high/low/centre point of the next candle that is 5 candles ahead, or predict the next 3 candles from 5 candles ahead onwards etc.

This is a simple multi-layer perceptron - it isn't great. Haven't experimented with a GRU or LSTM, but that'll be next. What could something like this be used for? Would the only use be to feed into a bigger network for position strategies?

In the backtest catalog I work with, every strategy carries a Deflated Sharpe score (Bailey and Lopez de Prado). The part that surprises people: a strategy's score can drop without any change to its own returns, just because more variants of other ideas got tested.

I think of it as plain selection. Test 20 versions of an idea with no edge and the best one will still show a decent Sharpe by luck. The Deflated Sharpe asks whether your winner beats the Sharpe you'd expect from the luckiest of all your trials, given how long its record is and how fat its tails are. More trials raise that luck bar, so the count has to include every variant you ran, published or not. Ours counts every strategy variant we've backtested, retired and unpublished ones included. Plain single-fund holds are left out, since nobody searched for those.

History length matters as much as the Sharpe. In our safe-withdrawal table the top 6 rates sit between 14.34% and 19.64%. The first 5 have 18.5 to 52.6 years of history and clear the 0.90 bar. The 6th, at 14.34%, has 7.0 years and gets flagged fragile. Nearly the same headline number as the 5th (14.66%), very different evidence behind it.

Processing img 395vhzmrt0yh1...

Our count is still a lower bound. Lookbacks tried in a notebook before a variant got its own row aren't in it, and neither are the ideas other researchers dropped before publishing the paper a strategy came from.

When you count trials, do you include every parameter set you ran, or only the versions you saved as named models?

I'm building options bots in Nautilus Trader, and I need to know what other people use for options market data. We've already run into a problem. Claude Code pulled data from Interactive Brokers and locked out the bot connection. I'm concerned that once the bots go live, each one in its own Docker container, they'll lock each other out, and Claude sessions will too.

Questions: What options data source do your bots use, live and for backtests? Do you run one gateway per bot, or one shared feed? How do you keep containers and Claude sessions from knocking each other off the gateway? What does your data cost per month?

I’m working on a long-only ML stock-trading strategy and could use some outside perspective. The historical results looked promising, but the more recent results have been much weaker.

How it trades

It uses one-minute price/volume data, usually enters around 11:30 a.m.–12:30 p.m. ET, and closes everything at 3:55 p.m. No leverage or overnight holds.

It generally takes a small number of trades per day. There’s only one entry per stock per session, and each cash allocation gets used once that day.

I’ve tried engineered features, traditional ML, and neural networks. The current version averages predictions from several model fits.

Results (model | SPY)

- 2020–2024 research backtest | +157.4% | +96.4% |
- 2025 OOS validation | +6.8% | +17.7% |
- 2026 YTD simulated paper trading | −0.4% | +13.7% |

For 2020–2024, Sharpe was 0.90, beta to SPY 0.18, and max drawdown 20.2%. It beat SPY in three of five years.

The edge per trade is thin: roughly 0.1–0.15% net, depending on the period. Average winners are around +2.5%, and average losers around −2.25%. The backtest includes modeled transaction costs at twice our base assumptions.

The weakness also started before the OOS period: 2024 returned −10.6% versus SPY’s +24.9%. Averaging predictions made results less dependent on a particular model fit, but didn’t fix the weak periods.

My concern is that it might be picking relatively good stocks while the broader midday-to-close opportunity has deteriorated. An initial attempt to predict when that trading window would be favorable didn’t show useful skill.

What would you investigate first?

- What simple baseline would best separate stock-picking skill from just trading that time window?
- Would you revisit the fixed end-of-day exit, the training target, or the entry timing first?
- How would you test a “don’t trade today” rule without overfitting?
- With an edge this thin, which execution assumptions would you stress-test hardest?
- After extensive experimentation, what would make you stop researching this approach?

I also have a separate multiweek strategy with stronger historical growth but higher market exposure and drawdown. Combining them improves the historical portfolio metrics, but we want to understand whether the intraday strategy holds up on its own.

Would appreciate specific test ideas, useful controls, or a blunt assessment of what we might be missing.

Happy to share more information if needed.

While working on a custom python project, working on a theory, I stumbled upon something I didn’t expect. I can see when the market moving algos start. Starting with the very first trade the algo makes. The dats source includes every single trade made on the s&p500 + a few other tickers.

Right now I’m downloading every single trade of the s&p 500 utilizing 15 alpaca api keys at once to get more backtesting data. The backtesting data on TradingView doesn’t even come close to cutting it. I made an app that does them all over 15 proxies, formats them in the format I need, does the calculations on my 5090 GPU and creates the json files and database for the gui. Zips the original files as well for backup. It does one full day of data in 3 mins. My bottleneck at the moment is my server provider which limits me to 100gb uploaded per day.

What I’m hoping to get some help with here is pointers on what to look for in terms of patterns of the algos on this backtesting. I can use AI to control it and want to try and steer it in the right direction. Since the data includes every single trade, I can see down to the millisecond.

Short summary of my current situation: I constructed a trading strategy which took me around 18 months and close to 1000 hours to develop. Most of my weekends and weekday evenings were spent on this. First 500-600 hours were literally just spent looking at charts and observing/documenting what's relevant and what's not. The other 400 - 500 hours were spent putting it on paper, and creating the algorithm + later testing.

Independent backtests via the API connected to IB's paper account were very successful. Tested over several months, accounting for commissions and spreads. Average number of trades was 150 per month or around 7 per day.

Here's the problem: the strategy relies heavily on 1-second data, meaning even a 1-second difference in an entry or exit can have quite a big impact on profits. At 9:30:58 ET, it takes a fixed screener snapshot of NYSE and NASDAQ-listed stocks. That snapshot is evaluated at around 9:31:05 to allow some time for data processing and for candles from 9:30:58 to finalize. Usually, around 50–70 stocks pass the screener each day. All of those stocks then need to be continuously monitored. Execution timing is extremely important, especially for exits.

IB doesn't allow screening that many stocks simultaneously, so I need a different data provider. I tested Alpaca SIP, but the differences compared to IB's historical data for both pricing and volume are substantial. Some of those differences actually change trading signals.

I read that IB's paper trading data is not always the most reliable, but also read about users having negative experiences with Alpaca. Maybe both are not the most reliable tools here. Maybe if I would live test (so not on via Paper trading), then the discrepancies would disappear or become manageable. I'm considering Massive, but fear that I'll run into the same problems which I encountered with Alpaca.

Any suggestions?

Hi! I'll keep this short and pls pls answer if you're seeing this. This will help me a lot.

I am genuinely interested in finance like genuinely. I love reading about and around it, have been looking at the technical aspects n algo trading now.

I am an engineering student (20) and I would love to work in the domain but seeing AI being able to do almost everything has gotten me shivering and confused whether I should pursue this path or not and get into model training or infra as my main thing.

I would be really grateful if I can get yall's advice on this

Tracking Trump's trades has always been annoying. The filings are scattered, hard to read, and most sites that clean them up put them behind a paywall.

So I added them to StockNest, and it's completely free:

- No paywall and no sign-up. Just open the page.

- Shows up as soon as it's published. New disclosures are added the moment they're filed.

- Easy to read. Each trade shows the ticker, the transaction size and the share amount.

- Filter by trade size. Show only the bigger trades, like $250K+, so the small ones are out of the way.

Donald J. Trump trades: https://stocknest.app/filers/politicians/donald-trump

Let's assume I have already developed a strategy that generates market trend signals, classifying the market into categories such as:

• Strong Bullish
• Bullish
• Neutral
• Bearish
• Strong Bearish

Now, I want to verify whether these signals correctly represent the actual market trend.

One straightforward approach would be to manually inspect every signal on a price chart and check whether the predicted trend matches the actual market movement. However, this is practically impossible at my scale.

I'm working with approximately 4,000 instruments, with around 100,000 signals per instrument. That means I could have to validate hundreds of millions of signals.

My question is: What methodology can I use to objectively measure the accuracy of these already-generated trend signals at scale?

I'm not looking for suggestions on how to improve my strategy, which indicators to use, or how to generate better signals. Please assume that the signals are already generated and I only need to evaluate their correctness against actual market data.

Ideally, I'd like a systematic or automated approach that can tell me how often each trend classification was correct, incorrect, or somewhere in between.

I'd really appreciate any insights, research papers, open-source tools, or methodologies that could help with this problem.

Thanks in advance!

How can algo trading be done in the Indian setup, and has anyone done it successfully?

Recent Grad, I'm a bit scared of F&Os in the Indian stock market, so I want to stick to equities/ stocks for now.

I also wanted to ask:

• Long-term positions: Can we use algorithmic strategies to find and manage long-term positions in cash equities?
• Minimum capital: What would you suggest as a realistic minimum capital to start with?

I checked via ChatGPT and it suggested that tools like Ollama can run data on a locally run AI/LLM (like Gemma) for free. Is that correct, and can you provide some guidance on this? What is the minimal locally run setup required?

With tools like Claude and other AI assistants, I want to try building something simple, learn from the process, and experiment. Any guidance or advice would be greatly appreciated!

~ structured properly using AI

edits:

~ recent grad as in I have job as SWE (6 months exp) I'm in Golang but with AI python will be easy to code

~ I can arrange 1-1.5L - that's the min average from responses

~ If any starting point you guys would suggest would be so much better.

thanks for responding! 🙏

Hi,

I'm always strengthening my auto trader and curious about what people prefer, and why. So wondering do users predominately use algotraders for stocks or crypto? If there is a clear leader between the two, can I please get some pros and cons of each from a user experience?

Any advice, suggestions, etc are welcomed.

Hi everyone i hop you're doing well !

like the title said i was wondering on how do you guys actually think/generate strategies hypothesis, how your approach of this process is, do you rely on some specific techniques, do you start by a random idea then keep optimizing, or you set a clear hypothesis with all the rules you want, i want to know exactly how you done the process of strategy hypothesis generating/thinking, the steps you pass, and also how the frequency of strategies hypothesis you backtest often in a day, do you rely heavily on optimizing the rules of the same strategy (hypothesis) or you have another technique

because litterally i face somehow a problem, simple stupid ideas like "9:00 AM ORB" or "Buy everyday if the price do this" are always have results equal or worse than a coin flip decision, i'm talking about breakeven or slightly worse on Gross results, but after costs it turn to complete disaster

i would love to hear some responses from you guys, because the strztegy hypothesis is the first thing you do before even open you backtester, a bad approach of strategy hypothesis generating will lead to waste months without having 1 profitable and robust strategy that passes your filters

I’m using ChatGPT to help build a backtest using 10 years of minute-by-minute SPX options data downloaded from **ThetaData**. I’d appreciate feedback on both the data and my approach.

The way I’ve been prompting ChatGPT is roughly:

> Look for any strategy that could have produced at least a 1,500% cumulative return over 10 years with a maximum drawdown below 40%.

These are search targets, not results I’ve verified. I realize that asking it to search for anything that meets a specific return and drawdown target could encourage overfitting or cherry-picking. I want to understand how to structure the process so the results are meaningful.

A few questions:

- **ThetaData quality:** Has anyone used its historical SPX options data? What checks do you recommend for missing or stale quotes, timestamp alignment, and correctly distinguishing contracts and settlement types?
- **Midpoint vs. bid/ask:** How realistic is assuming midpoint fills? Should I compare that against buying at the ask and selling at the bid, with commissions and slippage included? How would you handle multi-leg fills and wide spreads?
- **Strategy searching:** How would you structure training, validation, walk-forward testing, and an untouched test period when searching across many strategies? How do you account for the number of strategies and parameter combinations tried?
- **Return and drawdown calculations:** What should I check around position sizing, buying power, compounding, and marking open positions? I want the drawdown calculation to include losses while trades are open.
- **ChatGPT pitfalls:** What mistakes should I audit for in AI-generated backtest code—especially look-ahead bias, unrealistic fills, or code that runs successfully but implements different rules than intended?

Is this a reasonable research workflow, and how would you change the prompt or validation process before trusting anything it finds?

If there is an API or place I could scrape from daily OHLC that would be great.

Has anyone come across a period of time when your algorithm performed perfectly but fails in the backtest? I thought this was super weird.

From Sept 20 to Oct 18 I've done about 2-6% daily. And on the backtests it blows up the account.

For my backtesting, I tested S&P 500 index, its futures and ETFs e.g. ES, SPY

I did expect some performance variance since indices are not tradable, but I didn't expect the difference to be so big.

For example, with the same timeframe, setup and trading hours, the profit factor of S&P 500 index is 1.6, but it drops to 1.1 for ES. I expected them to have similar price movements, and I didn't think the difference would be that big.

Just curious - has anyone experienced this issue before? Any ways to reduce the difference? Thanks in advance