Quants see promise in DeBerta’s untangled reading

Improved language models are able to grasp context better

Context, as they say, is everything – which is a big problem for investors when they try to use so-called large language models to weigh the sentiment of financial news. The models are notorious for misreading terms that could be either good or bad depending on what’s being talked about at the time.

A few methods have been tried to solve the problem, mostly using the idea that models can look at the words around those they want to make sense of. One such model, FinBert, a version of Google’s open