Seminars / Conferences

Over the last two decades, researchers in operations management have increasingly leveraged laboratory experiments to identify key behavioral insights. These experiments inform behavioral theories of operations management, impacting domains including inventory, supply chain management, queuing, forecasting, and sourcing. Yet, until now, the replicability of most behavioral insights from these laboratory experiments has been untested. We remedy this with the first large-scale replication study in operations management. With the input of the wider operations management community, we identify 10 prominent experimental operations management papers published in Management Science, which span a variety of domains, to be the focus of our replication effort. For each paper, we conduct a high-powered replication study of the main results across multiple locations using original materials (when available and suitable). In addition, our study tests replicability in multiple modalities (in-person and online) due to laboratory closures during the COVID-19 pandemic. Our replication study contributes new knowledge about the robustness of several key behavioral theories in operations management and contributes more broadly to efforts in the operations management field to improve research transparency and reliability.

Randomized experiments have become the standard method for companies to evaluate the performance of new products or services. In addition to augmenting managers’ decision-making, experimentation mitigates risk by limiting the proportion of customers exposed to innovation. Since many experiments are on customers arriving sequentially, a potential solution is to allow managers to “peek” at the results when new data becomes available and stop the test if the results are statistically significant. Unfortunately, peeking invalidates the statistical guarantees for standard statistical analysis and leads to uncontrolled type-1 error. Our paper provides valid design-based confidence sequences, sequences of confidence intervals with uniform type-1 error guarantees over time for various sequential experiments in an assumption-light manner. In particular, we focus on finite-sample estimands defined on the study participants as a direct measure of the incurred risks by companies. Our proposed confidence sequences are valid for a large class of experiments, including multi-arm bandits, time series, and panel experiments. We further provide a variance reduction technique incorporating modeling assumptions and covariates. Finally, we demonstrate the effectiveness of our proposed approach through a simulation study and three real-world applications from Netflix. Our results show that by using our confidence sequence, harmful experiments could be stopped after only observing a handful of units; for instance, an experiment that Netflix ran on its sign-up page on 30,000 potential customers would have been stopped by our method on the first day before 100 observations.

Two-sided platforms typically generate revenue by matching prospective buyers and sellers and extracting commissions from completed transactions. Disintermediation, where sellers transact offline with buyers to bypass commission fees, can undermine the viability of these platforms. While transacting offline allows sellers to avoid commission fees, it also leaves them fully exposed to risky buyers given the absence of the platform’s protections. In this paper, we examine how disintermediation and information quality – specifically, the accuracy of the signal sellers receive about a buyer’s type – jointly impact a platform’s revenue and optimal commission rate. In a setting where transactions occur online-only, an increase in information quality leads sellers to set more efficient prices and complete more transactions, which lifts platform revenue. However, if sellers can transact offline, this effect may be reversed – additional information about buyers can hurt platform revenue by reducing the risk sellers face offline, amplifying disintermediation. Further, while intuition suggests platforms should counter disintermediation by lowering commission rates, in a high-information environment a platform may be better off raising them. Lastly, while charging sellers platform-access fees can hedge against losses from disintermediation, it can fall short of the revenue attained under commission-based pricing when all transactions occur online. Overall, our findings provide insight into the mechanisms through which disintermediation disrupts platform operations and offers prescriptions to platforms seeking to counteract it.

Online allocation problems with resource constraints are central problems in revenue management and online advertising. In these problems, requests arrive sequentially during a finite horizon, and for each request, a decision maker needs to choose an action that consumes a certain amount of resources and generates a reward. The objective is to maximize cumulative rewards subject to a constraint on the total consumption of resources. In practice, the distributions of reward and resource consumption are unknown, and there is a need to design data-driven algorithms that can make decisions in uncertain environments.

In this talk, we overview a new algorithm for online allocation problems, called dual mirror descent, that is simple, robust, and flexible. Compared to existing approaches, dual mirror descent is faster as it does not require solving auxiliary optimization problems, is more flexible because it can handle many applications across different sectors with minimal modifications, and is more robust as it enjoys remarkable performance under different environments without knowing which environment it is facing. In particular, dual mirror descent is asymptotically optimal under independent and identically distributed inputs as well as various nonstationary stochastic input models, and it attains an asymptotically optimal fixed competitive ratio when the input is adversarial. We discuss several extensions and applications to network revenue management, online bidding in repeated auctions with budget constraints, personalized assortment optimization with limited inventory, and fair allocation of resources.

The talk is based on the following papers:

https://pubsonline.informs.org/doi/abs/10.1287/opre.2021.2242

https://proceedings.mlr.press/v119/balseiro20a.html

https://proceedings.mlr.press/v139/balseiro21a.html

https://proceedings.mlr.press/v202/balseiro23a.html

https://dl.acm.org/doi/abs/10.1145/3578338.3593559

Forthcoming

Forthcoming