English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the recent advances showing that a model pre-trained on large-scalesource code data is able to gain appreciable generalization capability, itstill requires a sizeable amount of data on the target task for fine-tuning.And the effectiveness of the model generalization is largely affected by thesize and quality of the fine-tuning data, which is detrimental for target taskswith limited or unavailable resources. Therefore, cross-task generalization,with the goal of improving the generalization of the model to unseen tasks thathave not been seen before, is of strong research and application value.  In this paper, we propose a large-scale benchmark that includes 216 existingcode-related tasks. Then, we annotate each task with the corresponding metainformation such as task description and instruction, which contains detailedinformation about the task and a solution guide. This also helps us to easilycreate a wide variety of ``training/evaluation'' task splits to evaluate thevarious cross-task generalization capabilities of the model. Then we performsome preliminary experiments to demonstrate that the cross-task generalizationof models can be largely improved by in-context learning methods such asfew-shot learning and learning from task instructions, which shows thepromising prospects of conducting cross-task learning research on ourbenchmark. We hope that the collection of the datasets and our benchmark willfacilitate future work that is not limited to cross-task generalization.

CrossCodeBench: Benchmarking Cross-Task Generalization of Source Code Models