Probabilistic Modeling for Sequences of Sets in Continuous-Time

Neural marked temporal point processes have been a valuable addition to theexisting toolbox of statistical parametric models for continuous-time eventdata. These models are useful for sequences where each event is associated witha single item (a single type of event or a "mark") -- but such models are notsuited for the practical situation where each event is associated with a set ofitems. In this work, we develop a general framework for modeling set-valueddata in continuous-time, compatible with any intensity-based recurrent neuralpoint process model. In addition, we develop inference methods that can usesuch models to answer probabilistic queries such as "the probability of item$A$ being observed before item $B$," conditioned on sequence history. Computingexact answers for such queries is generally intractable for neural models dueto both the continuous-time nature of the problem setting and thecombinatorially-large space of potential outcomes for each event. To addressthis, we develop a class of importance sampling methods for querying withset-based sequences and demonstrate orders-of-magnitude improvements inefficiency over direct sampling via systematic experiments with four real-worlddatasets. We also illustrate how to use this framework to perform modelselection using likelihoods that do not involve one-step-ahead prediction.