small models can generalize well and outperform close big models in more and more tasks
small models can generalize well and outperform close big models in more and more tasks
they are be extremely quick & cheap to train (so many good tools for fine-tuning). data quality and good eval are important. what qualifies as good still needs good human judgement to be encoded into rules.
this produces more layers for businesses to inhabit and more specialized and reusable open models.
i see a small, light, and local future