CaTS-Bench: Can Language Models Describe Time Series?

arXiv (Cornell University) · 2025-09-25

Time series captioning, the task of describing time series in natural language, requires numeric and temporal reasoning, trend interpretation, and contextual understanding. Existing benchmarks, however, often rely on fully synthetic or generic captions, and typically neglect metadata and visual representations. We introduce CaTS-Bench, a comprehensive benchmark for Context-aware Time Series reasoning across 11 diverse domains, centered on a gold-standard evaluation set of 1746 human-rewritten captions that measure how effectively models translate numeric trends into immediately interpretable narratives. To address the scarcity of human-annotated data, we also propose a scalable pipeline for generating high-fidelity synthetic captions, the quality of which we validate. We evaluate leading Vision-Language Models on our benchmark, revealing that even proprietary models struggle to capture numeric nuances in temporal descriptions, while finetuning open-source models on synthetic data yields substantial performance gains. Finally, we release a diagnostic suite of 910 multiple-choice questions and use tailored numeric metrics to gauge time-series-specific reasoning capabilities, establishing CaTS-Bench as a reliable foundation for grounded, multimodal text generation in numeric domains.

The Value of Social Media Data in Fashion Forecasting

Manufacturing & Service Operations Management · 2023 · 25 citations

• Computer Science
• Marketing
• Computer Science

Problem definition: How to use social media to predict style color and jeans fit sales for a retailer. Academic/practical relevance: Neither retail practice nor the academic literature provides a method for using social media to predict style color and jeans fit sales for a retailer. We present and validate a systematic approach for doing that. Methodology: Demand forecasting in the fashion industry is challenging due to short product lifetimes, long manufacturing lead times, and constant innovation of fashion products. We investigate the value of social media information for color trends and jeans fit forecasting. We partner with three multinational retailers, two apparel and one footwear, and combine their proprietary data sets with web-crawled publicly available data on Twitter and the Google Search Volume Index. We implement a variety of machine learning models to develop forecasts that can be used in setting the initial shipment quantity for an item, arguably the most important decision for fashion retailers. Results: Our findings show that fine-grained social media information has significant predictive power in forecasting color and fit demands months in advance of the sales season, and therefore greatly helps in making the initial shipment quantity decision. The predictive power of including social media features, measured by the improvement of the out-of-sample mean absolute deviation over current practice ranges from 24% to 57%. Managerial implications: To our knowledge, this study is the first to explore and demonstrate the value of social media information in fashion demand forecasting in a way that is practical and operable for fashion retailers. With consistent results across all three retailers, we demonstrate the robustness of our findings over market and geographic heterogeneity, and different forecast horizons. Moreover, we discuss potential mechanisms that might be driving this significant predictive power. Our results suggest that changes in fashion demand are driven more by “bottom-up” changes in consumer preferences than by “top-down” influence from the fashion industry. Funding: This work was supported by Wharton School Fishman-Davidson Center for Service and Operations Management, the Wharton School Baker Retailing Center, and the Wharton School Risk Management Center Russell Ackoff Doctoral Student Fellowship. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.1193 .

Innovations in retail operations: Thirty years of lessons from <i>Production and Operations Management</i>

Production and Operations Management · 2022-09-10 · 11 citations

We review papers published in POM during its thirty‐year history that deal with retail operations issues with an empirical approach. The papers span a range of issues, from traditional ones like forecasting and inventory planning, to new technologies, like radio frequency identification (RFID) and e‐commerce, and strategic, like links between retailing and stock market performance.

Setting Retail Staffing Levels: A Methodology Validated with Implementation

Manufacturing & Service Operations Management · 2020 · 24 citations

• Computer Science
• Business
• Operations management

Problem definition: How should retail staffing levels be set? While cost of labor is well understood, the revenue implications of having the right staffing level are hard to estimate. Moreover, these implications vary by store; hence, staffing levels should vary as well. Academic/practical relevance: We provide a novel method for setting store associate staffing at the individual store level. We discuss a field implementation that tested this methodology. Methodology: We use historical data on revenue and planned and actual staffing levels by store to estimate how revenue varies with the staffing level at each store. We disentangle the endogeneity between revenue and staffing levels by focusing on randomly occurring deviations between planned and actual labor. Using historical analysis as a guide, we validate these results by changing the staffing levels in a few test stores. We implement the results chain-wide and measure the impact in a large specialty retailer. Results: We find that the implementation validates predictions of the historical analysis. The implementation in 168 stores over six months produces a 4.5% revenue increase and a nearly $7.4 million annual profit increase. The impact of staffing level on revenue varies greatly by store. Managerial implications: Our paper makes three contributions to academic literature and to retail practice. First, we describe a process by which retailers can improve the most common industry practice: set store labor to be proportional to forecasted store revenue. Our proposed approach systematically sets the labor level in each store. Second, we demonstrate the effectiveness of that process via a field test and then via chain-wide implementation over a six-month time period. Finally, most retailers set store labor at the same level across stores, proportionate to revenue. We show that this is not the best approach because the revenue impact of store labor varies by store. The stores in our study that could benefit from relatively more labor were those with high potential demand, closely located competition for that demand, and experienced store managers. Overall, we provide the first simple but rigorous, field-tested approach that any retailer can use to increase revenue and profitability through better labor management.

Does Online Training Work in Retail?

Manufacturing & Service Operations Management · 2020 · 16 citations

• Computer Science
• Marketing
• Business

Problem definition: How much, if at all, does training in product features increase a sales associate’s sales productivity? Academic/practical relevance: A knowledgeable retail sales associate (SA) can explain the features of available product variants and give a customer sufficient confidence in the customer’s choice or suggest alternatives so that the customer becomes willing to purchase. Although it is plausible that increasing an SA’s product knowledge will increase sales, training is not without cost and turnover is high in retail, so most retailers provide little product-knowledge training. Methodology: We partner with two firms and collect data on more than 50,000 SAs who had access to training. We assemble a detailed data set of the training history and individual sales productivity over a two-year period. We conduct econometric analysis to quantify the causal effect of training on sales. Results: For SAs who engaged in training, the sales rate increases by 1.8% for every online module taken, which is a much higher benefit than the direct or indirect costs associated with this training. Brand-specific training has a larger effect on the focal brand; however, there is a positive effect on other brands the SA sells. We also assess how the training benefit varies depending on the SA’s tenure, sales rate prior to training, and number of modules taken. Managerial implications: We present evidence of a novel training mechanism that can be extremely attractive to retailers. Online training tools, such as the one we study, have two characteristics that should not be overlooked. First, it is the brands, not the retailers, that create, develop, and pay for the training content. Second, the incentives are such that SAs invest their own time, rather than time on the job, to train, and this makes the retailer’s investment in the training a profitable proposition.

Optimal Retail Location: Empirical Methodology and Application to Practice

Manufacturing & Service Operations Management · 2019-01-01 · 110 citations

We empirically study the spatiotemporal location problem motivated by an online retailer that uses the Buy-Online-Pick-Up-In-Store fulfillment method. Customers pick up their orders from trucks parked at specific locations on specific days, and the retailer’s problem is to determine where and when these pickups occur. Customer demand is influenced by the convenience of pickup locations and days. We combine demographic and economic data, business location data, and the retailer’s historical sales and operations data to predict demand at potential locations. We introduce a novel procedure that combines machine learning and econometric techniques. First, we use a fixed effects regression to estimate spatial and temporal cannibalization effects. Then, we use a random forests algorithm to predict demand when a particular location operates in isolation. Based on the predicted demand and cannibalization effects, we solve the spatiotemporal integer program using a quadratic program relaxation to find the optimal pickup location configuration and schedule. We estimate a revenue increase of at least 51% from the improved location configuration and schedule. The online appendices are available at https://doi.org/10.1287/msom.2018.0759 .

Why Empirical Research Is Good for Operations Management, and What Is Good Empirical Operations Management?

Manufacturing & Service Operations Management · 2019-11-08 · 49 citations

Empirical research in operations management has increased steadily over the last 20 years. In this paper, we discuss why this is good for our field and offer some comments on the qualities we admire in an empirical operations management paper.

The Value of Rapid Delivery in Omnichannel Retailing

Journal of Marketing Research · 2019-07-03 · 190 citations

The authors study how faster delivery in the online channel affects sales within and across channels in omnichannel retailing. The authors leverage a quasi-experiment involving the opening of a new distribution center by a U.S. apparel retailer, which resulted in unannounced faster deliveries to western U.S. states through its online channel. Using a difference-in-differences approach, the authors show that online store sales increased, on average, by 1.45% per business-day reduction in delivery time, from a baseline of seven business days. The authors also find a positive spillover effect to the retailer’s offline stores. These effects increase gradually in the short-to-medium run as the result of higher order count. The authors identify two main drivers of the observed effect: (1) customer learning through service interactions with the retailer and (2) existing brand presence in terms of online store penetration rate and offline store presence. Customers with less online store experience are more responsive to faster deliveries in the short run, whereas experienced online store customers are more responsive in the long run.

The Value of Rapid Delivery in Omnichannel Retailing

Sage Journals Data · 2019-01-01 · 23 citations

The authors study how faster delivery in the online channel affects sales within and across channels in omnichannel retailing. The authors leverage a quasi-experiment involving the opening of a new distribution center by a U.S. apparel retailer, which resulted in unannounced faster deliveries to western U.S. states through its online channel. Using a difference-in-differences approach, the authors show that online store sales increased, on average, by 1.45% per business-day reduction in delivery time, from a baseline of seven business days. The authors also find a positive spillover effect to the retailer’s offline stores. These effects increase gradually in the short-to-medium run as the result of higher order count. The authors identify two main drivers of the observed effect: (1) customer learning through service interactions with the retailer and (2) existing brand presence in terms of online store penetration rate and offline store presence. Customers with less online store experience are more responsive to faster deliveries in the short run, whereas experienced online store customers are more responsive in the long run.

Using Data and Big Data in Retailing

Production and Operations Management · 2018-01-10 · 83 citations

In this essay, we examine how retailers can use data to make better decisions and hence, improve their performance. We have been studying retail operations for over two decades and have witnessed many, and been personally involved in a few, projects that delivered considerable value to retailers by better exploiting data. We highlight a few of these examples and also identify some other potential applications.