Datasets:
|
Download README.md from XEUIPR/Java-Code-Large-text-only: direct link, hf CLI and curl.
- Browser
- Download file 2.28 kB
-
https://huggingface.co/datasets/XEUIPR/Java-Code-Large-text-only/resolve/main/README.md
- Command line
-
hf download hf://datasets/XEUIPR/Java-Code-Large-text-only/README.md
-
curl -L -o README.md https://huggingface.co/datasets/XEUIPR/Java-Code-Large-text-only/resolve/main/README.md
2.28 kB
| license: mit | |
| task_categories: | |
| - text-generation | |
| language: | |
| - en | |
| tags: | |
| - code | |
| - java | |
| size_categories: | |
| - 10M<n<100M | |
| **Java-Code-Large** | |
| Java-Code-Large is a large-scale corpus of publicly available Java source code comprising more than **15 million** java codes. The dataset is designed to support research in large language model (LLM) pretraining, code intelligence, software engineering automation, and program analysis. | |
| By providing a high-volume, language-specific corpus, Java-Code-Large enables systematic experimentation in Java-focused model training, domain adaptation, and downstream code understanding tasks. | |
| **1. Introduction** | |
| Large-scale code corpora have become fundamental resources for training and evaluating machine learning models for code-related tasks. While multilingual code datasets exist, there is increasing interest in language-specialized corpora to: | |
| - Improve domain-specific performance | |
| - Reduce cross-language noise | |
| - Enable controlled experimental settings | |
| - Support Java-specific tooling and research | |
| Java-Code-Large addresses this need by providing a dedicated Java-only dataset at substantial scale. | |
| **2. Dataset Composition** | |
| Programming Language: Java | |
| File Count: 15M+ Java files | |
| File Format: .jsonl | |
| Content Types: | |
| - Classes | |
| - Interfaces | |
| - Enums | |
| - Methods | |
| - Annotations | |
| - JavaDoc comments | |
| - Exception handling structures | |
| - Generics and concurrency constructs | |
| The dataset consists of source code extracted from publicly accessible open-source repositories. | |
| **3. Intended Research Applications** | |
| 3.1 Pretraining | |
| - Training code foundation models from scratch | |
| - Continued pretraining of existing LLMs | |
| - Java-specialized language modeling | |
| 3.2 Fine-Tuning and Adaptation | |
| - Code completion systems | |
| - Automated refactoring tools | |
| - IDE copilots | |
| - Java-specific conversational assistants | |
| 3.3 Code Intelligence Tasks | |
| - Code summarization | |
| - Code-to-text generation | |
| - Bug detection | |
| - Vulnerability detection | |
| - Clone detection | |
| - Code similarity modeling | |
| - Static and structural analysis | |
| 3.4 Software Engineering Research | |
| - Empirical studies of Java programming patterns | |
| - Tokenization and AST modeling experiments | |
| Thanks to open source community for all the guidance & support!! | |