An Improvement of Choosing Map-join Candidates in Hive

Currently Hive uses the size of tables on disk to determine if a common-join should be converted into a map-join. Our experiments demonstrate that this is a conservative decision criteria which will fail to identify optimization opportunities close to the decision frontier. Our implementation differs from the current implementation in the way we identify optimization candidates. By pre-computing hashtable sizes and adjusting them with per-query selectivity factors, we are able to choose optimization candidates directly as a function of expected hashtable size, factoring out the conservative file size criteria. In this paper we show that this approach results in more common-join to map-join optimizations and can provide average speedups of up to 1.30 in certain scenarios