•1 min read•from Machine Learning
Why do only big ML labs dominate widely-used models despite many open-source pretrained models smaller labs could do RL on? [D]
I’m trying to understand why models from major labs (GPT, Claude, etc.) dominate real-world usage? You might say it's due to the expensive pretraining compute budge, but there already exists many pretrained open-source models at the same scale (e.g., Kimi).
Of course Kimi isn't as good as Claude, but it's the RL on top of the pretraining that makes Claude what it is right? Given Kimi, DeepSeek etc all have the expensive pretraining done, the RLHF on top is what makes Claude what it is right? And that should be much more accessible in terms of cost to smaller labs no?
[link] [comments]
Want to read more?
Check out the full article on the original site
Tagged with
#rows.com
#real-time data collaboration
#real-time collaboration
#big data management in spreadsheets
#big data performance
#RLHF (Reinforcement Learning from Human Feedback)
#ML labs
#RL (Reinforcement Learning)
#Claude
#open-source pretrained models
#widely-used models
#pretrained models
#machine learning
#pretraining compute budget
#expensive pretraining
#real-world usage
#Kimi
#training data
#model performance
#DeepSeek