🦾 微调 LLM 和其他语言模型#

反馈数据集#

注意

本节介绍的数据集类是 FeedbackDataset。这个完全可配置的数据集将在 Argilla 2.0 中取代 DatasetForTextClassification、DatasetForTokenClassification 和 DatasetForText2Text。不确定使用哪个数据集？请查看我们关于选择数据集的部分。

在从我们的 FeedbackDataset 收集回复后，我们可以开始微调我们的 LLM 和其他模型。由于任务的可定制性，这可能需要设置自定义的后处理工作流，但我们将为 LLM 方法提供一些好的示例：监督微调和通过人类反馈的强化学习 (RLHF)。然而，我们仍然为其他 NLP 任务（如文本分类）提供支持。

`ArgillaTrainer`#

ArgillaTrainer 是对我们许多喜爱的 NLP 库的封装。它提供了一个非常直观的抽象表示，以方便使用合理的默认预设配置进行简单的训练工作流，而无需担心来自 Argilla 的任何数据转换。

使用 ArgillaTrainer 很简单，但每个任务略有不同。

首先，我们定义一个 TrainingTask。这可以使用自定义的 formatting_func 完成。然而，像文本分类这样的任务也可以使用默认定义来定义，使用 FeedbackDataset 字段和问题。这些任务然后用于从数据集中检索数据并初始化训练。我们还为开箱即用的统一数据提供了一些想法。
接下来，我们初始化 ArgillaTrainer 并转发任务和训练框架。在内部，这使用 FeedbackData.prepare_for_training 方法来根据框架的期望格式化数据。一些其他有趣的方法是
1. ArgillaTrainer.update_config 用于更改特定于框架的训练参数。
2. ArgillaTrainer.train 用于开始训练。
3. ArgillTrainer.predict 用于运行推理。

在下面，您可以看到使用 ArgillaTrainer 的愉快流程。

from argilla.feedback import ArgillaTrainer, FeedbackDataset, TrainingTask

dataset = FeedbackDataset.from_huggingface(
    repo_id="argilla/emotion"
)
task = TrainingTask.for_text_classification(
    text=dataset.field_by_name("text"),
    label=dataset.question_by_name("label"),
)
trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit"
)
trainer.update_config(num_iterations=1)
trainer.train(output_dir="my_setfit_model")
trainer.predict("This is awesome!")

支持的框架#

我们计划增加对其他任务和框架的支持，因此请随时通过我们的 Discord 频道或 GitHub 与我们联系，以帮助我们确定每个任务的优先级。

任务/框架	TRL	OpenAI	SetFit	spaCy	Transformers	PEFT	SentenceTransformers
文本分类			✔️	✔️	✔️	✔️
问题回答					✔️
句子相似度							✔️
监督微调	✔️
奖励建模	✔️
近端策略优化	✔️
直接偏好优化	✔️
聊天完成		✔️

训练配置#

训练器还有一个 ArgillaTrainer.update_config() 方法，它将带有 **kwargs 的字典映射到相应的框架。因此，这些可以从用于初始化训练器的底层框架派生出来。在下面，您可以找到支持的框架的这些变量的概述。

注意

请注意，您不需要直接传递所有这些变量，并且下面的值是它们的默认配置。

OpenAI

# `OpenAI.FineTune`
trainer.update_config(
    training_file = None,
    validation_file = None,
    model = "gpt-3.5-turbo-0613",
    hyperparameters = {"n_epochs": 1},
    suffix = None
)

# `OpenAI.FineTune` (legacy)
trainer.update_config(
    training_file = None,
    validation_file = None,
    model = "curie",
    n_epochs = 2,
    batch_size = None,
    learning_rate_multiplier = 0.1,
    prompt_loss_weight = 0.1,
    compute_classification_metrics = False,
    classification_n_classes = None,
    classification_positive_class = None,
    classification_betas = None,
    suffix = None
)

AutoTrain

# `AutoTrain.autotrain_advanced`
trainer.update_config(
    model = "autotrain", # hub models like roberta-base
    autotrain = [{
        "source_language": "en",
        "num_models": 5
    }],
    hub_model = [{
        "learning_rate":  0.001,
        "optimizer": "adam",
        "scheduler": "linear",
        "train_batch_size": 8,
        "epochs": 10,
        "percentage_warmup": 0.1,
        "gradient_accumulation_steps": 1,
        "weight_decay": 0.1,
        "tasks": "text_binary_classification", # this is inferred from the dataset
    }]
)

SetFit

# `setfit.SetFitModel`
trainer.update_config(
    pretrained_model_name_or_path = "all-MiniLM-L6-v2",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `setfit.SetFitTrainer`
trainer.update_config(
    metric = "accuracy",
    num_iterations = 20,
    num_epochs = 1,
    learning_rate = 2e-5,
    batch_size = 16,
    seed = 42,
    use_amp = True,
    warmup_proportion = 0.1,
    distance_metric = "BatchHardTripletLossDistanceFunction.cosine_distance",
    margin = 0.25,
    samples_per_label = 2
)

spaCy

# `spacy.training`
trainer.update_config(
    dev_corpus = "corpora.dev",
    train_corpus = "corpora.train",
    seed = 42,
    gpu_allocator = 0,
    accumulate_gradient = 1,
    patience = 1600,
    max_epochs = 0,
    max_steps = 20000,
    eval_frequency = 200,
    frozen_components = [],
    annotating_components = [],
    before_to_disk = None,
    before_update = None
)

Transformers

# `transformers.AutoModelForTextClassification`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-uncased",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

Peft (LoRA)

# `peft.LoraConfig`
trainer.update_config(
    r=8,
    target_modules=None,
    lora_alpha=16,
    lora_dropout=0.1,
    fan_in_fan_out=False,
    bias="none",
    inference_mode=False,
    modules_to_save=None,
    init_lora_weights=True,
)
# `transformers.AutoModelForTextClassification`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-uncased",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

SpanMarker

# `SpanMarkerConfig`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-cased"
    model_max_length = 256,
    marker_max_length = 128,
    entity_max_length = 8,
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

TRL

# Parameters from `trl.RewardTrainer`, `trl.SFTTrainer`, `trl.PPOTrainer` or `trl.DPOTrainer`.
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

sentence-transformers

# Parameters related to the model initialization from `sentence_transformers.SentenceTransformer`
trainer.update_config(
    model="sentence-transformers/all-MiniLM-L6-v2",
    modules = False,
    device="cuda",
    cache_folder="dir/folder",
    use_auth_token=True
)
# and from `sentence_transformers.CrossEncoder`
trainer.update_config(
    model="cross-encoder/ms-marco-MiniLM-L-6-v2",
    num_labels=2,
    max_length=128,
    device="cpu",
    tokenizer_args={},
    automodel_args={},
    default_activation_function=None
)
# Related to the training procedure from `sentence_transformers.SentenceTransformer`
trainer.update_config(
    steps_per_epoch = 2,
    checkpoint_path: str = None,
    checkpoint_save_steps: int = 500,
    checkpoint_save_total_limit: int = 0
)
# and from `sentence_transformers.CrossEncoder`
trainer.update_config(
    loss_fct = None
    activation_fct = nn.Identity(),
)
# The remaining arguments are common for both procedures
trainer.update_config(
    evaluator: SentenceEvaluator = evaluation.EmbeddingSimilarityEvaluator,
    epochs: int = 1,
    scheduler: str = 'WarmupLinear',
    warmup_steps: int = 10000,
    optimizer_class: Type[Optimizer] = torch.optim.AdamW,
    optimizer_params : Dict[str, object]= {'lr': 2e-5},
    weight_decay: float = 0.01,
    evaluation_steps: int = 0,
    output_path: str = None,
    save_best_model: bool = True,
    max_grad_norm: float = 1,
    use_amp: bool = False,
    callback: Callable[[float, int, int], None] = None,
    show_progress_bar: bool = True,
)
# Other parameters that don't correspond to the initialization or the trainer, but
# can be set externally.
trainer.update_config(
    batch_size=8,  # It will be passed to the DataLoader to generate batches during training.
    loss_cls=losses.BatchAllTripletLoss
)

`TrainingTask`#

TrainingTask 用于定义应如何根据关联的任务和框架处理和格式化数据。每个任务都有自己的 TrainingTask.for_*-classmethod，并且数据格式化始终可以使用自定义的 formatting_func 定义。然而，像文本分类这样的简单任务也可以使用默认定义来定义。这些直接使用 FeedbackDataset 配置中的字段和问题来推断如何准备数据。在下面，您可以找到 TrainingTask 要求的概述。

方法	内容	`formatting_func` 返回类型	默认
for_text_classification	`文本-标签`	`Union[Tuple[str, str], Tuple[str, List[str]]]`	✔️
for_question_answering	`问题-上下文-答案`	`Union[Tuple[str, str], Tuple[str, List[str]]]`	✔️
for_sentence_similarity	`句子-1-句子-2-(句子-3)-(标签)`	`Union[Dict[str, Union[float, int]], Dict[str, str], List[Dict[str, Union[float, int]]], List[Dict[str, str]]]`	✔️
for_supervised_fine_tuning	`文本`	`Union[str, Iterator[str]]`	✗
for_reward_modeling	`选择-拒绝`	`Union[Tuple[str, str], Iterator[Tuple[str, str]]]`	✗
for_proximal_policy_optimization	`文本`	`Union[str, Iterator[str]]]`	✗
for_direct_preference_optimization	`提示-选择-拒绝`	`Union[Tuple[str, str, str], Iterator[Tuple[str, str, str]]]`	✗
for_chat_completion	`聊天轮次-角色-内容`	`Union[Tuple[str, str, str, str], Iterator[Tuple[str, str, str, str]]]`	✗

过滤和排序用于训练的数据集#

假设您想过滤数据集的一部分，仅保留已提交的记录，或者按日期排序以仅在数据集的最新添加项上进行训练。您可以通过使用 ArgillaTrainer 的 filter_by、sort_by 和 max_records 参数轻松完成此操作

from argilla import SortBy

trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit",
    filter_by={"response_status": ["submitted"]},
    sort_by=[SortBy(field="metadata.my-metadata", order="asc")],
    max_records=1000
)

注意

您可以查看文档中的过滤和查询数据集页面，以了解有关如何过滤和排序数据集的更多信息。

Huggingface Hub 集成#

本节介绍与 Hugging Face 🤗模型中心的一些集成，这是共享 Argilla 模型的最简单方法，以及生成自动化模型卡的可能性。

注意

查看 🤗huggingface hub 中带有自动生成模型卡的以下示例模型，并查看 https://hugging-face.cn/models?other=argilla 以获取即将推出的共享 Argilla 模型。

模型卡生成#

ArgillaTrainer 在保存模型时自动生成模型卡。在调用 trainer.train(output_dir="my_model") 后，您应该在通过 train 方法传递的相同输出目录下看到模型卡：./my_model/README.md。卡片中的大多数字段在可能的情况下会自动生成，但以下字段可以通过 ArgillaTrainer 的 framework_kwargs 变量（可选）更新，如下所示

model_card_kwargs = {
    "language": ["en", "es"],
    "license": "Apache-2.0",
    "dataset_name": "argilla/emotion",
    "tags": ["nlp", "few-shot-learning", "argilla", "setfit"],
    "model_summary": "Small summary of what the model does",
    "model_description": "An extended explanation of the model",
    "model_type": "A 1.3B parameter embedding model fine-tuned on an awesome dataset",
    "finetuned_from": "all-MiniLM-L6-v2",
    "repo": "https://github.com/..."
    "developers": "",
    "shared_by": "",
}

trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit",
    framework_kwargs={"model_card_kwargs": model_card_kwargs}
)
trainer.train(output_dir="my_model")

即使它是内部生成的，您也可以通过调用 generate_model_card 方法获取卡片

argilla_model_card = trainer.generate_model_card("my_model")

将您的模型上传到 Huggingface Hub#

如果您尚未安装 huggingface hub，可以使用以下命令进行安装

pip install huggingface_hub

注意

如果您选择的框架是 spacy 或 spacy-transformers，您还应该安装以下依赖项

pip install spacy-huggingface-hub

然后选择环境，具体取决于您是在脚本中还是在 jupyter notebook 中工作

控制台

从控制台窗口运行以下命令并插入您的 🤗huggingface hub 令牌

huggingface-cli login

Notebook

从 notebook 单元格运行以下命令并插入您的 🤗huggingface hub 令牌

from huggingface_hub import notebook_login

notebook_login()

在内部，令牌将在调用 push_to_huggingface 模型时使用。

如果您需要更多关于发布模型的帮助，请务必查看 huggingface hub 要求。

在您的模型训练完成后，您只需调用 push_to_huggingface 并等待您的模型被推送到 hub（默认情况下，将生成模型卡，如果您不想要它，请将参数设置为 False）

# spaCy based models:
repo_id = output_dir

# Every other framework:
repo_id = "organization/model-name"  # for example: argilla/newest-model

trainer.push_to_huggingface(repo_id, generate_card=True)

由于 spaCy 在推送模型时的行为，repo_id 会在内部自动生成，您需要传递模型保存的路径（您可能传递给 train 方法的相同 output_dir 变量），它将以相同的方式工作。

其他数据集#

注意

本节中涵盖的记录类对应于三个数据集：DatasetForTextClassification、DatasetForTokenClassification 和 DatasetForText2Text。这些将在 Argilla 2.0 中弃用，并由完全可配置的 FeedbackDataset 类取代。不确定使用哪个数据集？查看我们关于选择数据集的部分。

`ArgillaTrainer`#

支持的框架#

框架/任务	文本分类	Token 分类	Text2Text
OpenAI	✔️		✔️
SetFit	✔️
spaCy	✔️	✔️
Transformers	✔️	✔️
PEFT	✔️	✔️
SpanMarker		✔️

训练配置#

注意

请注意，您不需要直接传递所有这些变量，并且下面的值是它们的默认配置。

OpenAI

# `OpenAI.FineTune`
trainer.update_config(
    training_file = None,
    validation_file = None,
    model = "gpt-3.5-turbo-0613",
    hyperparameters = {"n_epochs": 1},
    suffix = None
)

# `OpenAI.FineTune` (legacy)
trainer.update_config(
    training_file = None,
    validation_file = None,
    model = "curie",
    n_epochs = 2,
    batch_size = None,
    learning_rate_multiplier = 0.1,
    prompt_loss_weight = 0.1,
    compute_classification_metrics = False,
    classification_n_classes = None,
    classification_positive_class = None,
    classification_betas = None,
    suffix = None
)

SetFit

# `setfit.SetFitModel`
trainer.update_config(
    pretrained_model_name_or_path = "all-MiniLM-L6-v2",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `setfit.SetFitTrainer`
trainer.update_config(
    metric = "accuracy",
    num_iterations = 20,
    num_epochs = 1,
    learning_rate = 2e-5,
    batch_size = 16,
    seed = 42,
    use_amp = True,
    warmup_proportion = 0.1,
    distance_metric = "BatchHardTripletLossDistanceFunction.cosine_distance",
    margin = 0.25,
    samples_per_label = 2
)

spaCy

# `spacy.training`
trainer.update_config(
    dev_corpus = "corpora.dev",
    train_corpus = "corpora.train",
    seed = 42,
    gpu_allocator = 0,
    accumulate_gradient = 1,
    patience = 1600,
    max_epochs = 0,
    max_steps = 20000,
    eval_frequency = 200,
    frozen_components = [],
    annotating_components = [],
    before_to_disk = None,
    before_update = None
)

Transformers

# `transformers.AutoModelForTextClassification`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-uncased",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

Peft (LoRA)

# `peft.LoraConfig`
trainer.update_config(
    r=8,
    target_modules=None,
    lora_alpha=16,
    lora_dropout=0.1,
    fan_in_fan_out=False,
    bias="none",
    inference_mode=False,
    modules_to_save=None,
    init_lora_weights=True,
)
# `transformers.AutoModelForTextClassification`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-uncased",
    force_download = False,
    resume_download = False,
    proxies = None,
    token = None,
    cache_dir = None,
    local_files_only = False
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

SpanMarker

# `SpanMarkerConfig`
trainer.update_config(
    pretrained_model_name_or_path = "distilbert-base-cased"
    model_max_length = 256,
    marker_max_length = 128,
    entity_max_length = 8,
)
# `transformers.TrainingArguments`
trainer.update_config(
    per_device_train_batch_size = 8,
    per_device_eval_batch_size = 8,
    gradient_accumulation_steps = 1,
    learning_rate = 5e-5,
    weight_decay = 0,
    adam_beta1 = 0.9,
    adam_beta2 = 0.9,
    adam_epsilon = 1e-8,
    max_grad_norm = 1,
    learning_rate = 5e-5,
    num_train_epochs = 3,
    max_steps = 0,
    log_level = "passive",
    logging_strategy = "steps",
    save_strategy = "steps",
    save_steps = 500,
    seed = 42,
    push_to_hub = False,
    hub_model_id = "user_name/output_dir_name",
    hub_strategy = "every_save",
    hub_token = "1234",
    hub_private_repo = False
)

任务#

在本部分中，我们将探讨文本分类、Token 分类和 Text2Text 任务。我们将简要描述每个任务的含义以及训练和进行预测所涉及的步骤。

文本分类#

背景#

文本分类是一种广泛使用的 NLP 任务，其中标签被分配给文本。主要公司依靠它来用于各种应用。情感分析是文本分类的一种流行形式，它将标签（如 🙂 正面、🙁 负面或 😐 中性）分配给文本。此外，我们区分单标签和多标签文本分类。

单标签

单标签文本分类是指为给定的文本样本分配单个类别或标签的任务。每个文本仅与一个预定义的类或类别相关联。例如，在情感分析中，单标签文本分类任务将涉及根据文本的情感为文本分配诸如“正面”、“负面”或“中性”之类的标签。

"The help for my application of a new card and mortgage was great", "positive"

多标签

由于确定和预测每个文本的多个相关标签的挑战，多标签文本分类通常比单标签分类更复杂。它在各个领域都有应用，包括文档标记、主题标记和内容推荐系统。例如，在客户服务中，多标签文本分类任务将涉及根据文本内容为文本分配诸如“new_card”、“mortgage”或“opening_hours”之类的主题。

提示

对于多标签场景，建议添加一些没有任何标签的示例，以提高模型性能。

"The help for my application of a new card and mortgage was great", ["new_card", "mortgage"]

训练#

from argilla.feedback import ArgillaTrainer, FeedbackDataset, TrainingTask

dataset = FeedbackDataset.from_huggingface(
    repo_id="argilla/emotion"
)
task = TrainingTask.for_text_classification(
    text=dataset.field_by_name("text"),
    label=dataset.question_by_name("label"),
)
trainer = ArgillaTrainer(
    dataset=dataset,
    task=task,
    framework="setfit"
)
trainer.update_config(num_iterations=1)
trainer.train(output_dir="my_setfit_model")
trainer.predict("This is awesome!")

Token 分类#

背景#

Token 分类是 NLP 领域中的一个关键概念。它需要为给定文本中的单个单词或 token 分配特定标签。这些标签可以包含各种语言或语义属性，例如词性标注、命名实体（包括人名、组织或地点）或情感指标（表达积极性、消极性或中立性）。此过程是许多 NLP 应用程序不可或缺的基础，有助于从文本数据中提取有价值的见解。

训练#

import argilla as rg
from datasets import load_dataset
from argilla.training import ArgillaTrainer

dataset_rg = rg.DatasetForTokenClassification.from_datasets(
    dataset=load_dataset("conll2003", split="train[:100]"),
    tags="ner_tags",
)
rg.log(dataset_rg, name="conll2003", workspace="admin")

trainer = ArgillaTrainer(
    name="conll2003",
    workspace="admin",
    framework="spacy",
    train_size=0.8
)
trainer.update_config(num_train_epochs=2)
trainer.train(output_dir="my_spacy_model")
records = trainer.predict("The ArgillaTrainer is great!", as_argilla_records=True)
rg.log(records=records, name="conll2003", workspace="admin")

Text2Text#

背景#

NLP 领域的 Text2Text 任务代表一个框架，该框架将一段文本作为输入以将其转换为另一段文本。 T2T 不是将不同的 NLP 挑战视为孤立的问题，而是试图通过将它们构建为序列到序列的转换来创建通用的解决方案。在这种方法中，输入和输出都被视为文本序列，并且它们的长度可能会有所不同。

训练#

import argilla as rg
from datasets import load_dataset
from argilla.training import ArgillaTrainer

dataset_rg = rg.DatasetForText2Text.from_datasets(
    dataset=load_dataset("opus_books", "en-fr", split="train[:100]"),
    tags="ner_tags",
)
rg.log(dataset_rg, name="opus_books", workspace="admin")

trainer = ArgillaTrainer(
    name="opus_books",
    workspace="admin",
    framework="openAI",
    train_size=0.8
)
trainer.update_config(max_epochs=2)
trainer.train(output_dir="my_openAI_model")
records = trainer.predict("The ArgillaTrainer is great!", as_argilla_records=True)
rg.log(records=records, name="opus_books", workspace="admin")

其他选项#

准备训练#

如果您想训练模型，我们提供了一种方便的方法来准备数据集：DatasetFor*.prepare_for_training()。它将返回一个 Hugging Face 数据集、一个 spaCy DocBin 或一个 SparkNLP 格式的 DataFrame，这些数据集、DocBin 或 DataFrame 针对使用 Hugging Face Trainer、spaCy CLI 或 SparkNLP API 进行的训练过程进行了优化。

可以通过传递 train_size 和 test_size 参数，直接将训练-测试拆分包含到 prepare_for_training 中。

OpenAI

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="openai", train_size=1)
# [{'promt': 'My title', 'completion': ' My content'}]

AutoTrain

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="autotrain", train_size=1)
# {'title': 'My title', 'content': 'My content', 'label': 0}

SetFit

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="setfit", train_size=1)
# {'title': 'My title', 'content': 'My content', 'label': 0}

spaCy

import argilla as rg
import spacy

nlp = spacy.blank("en")

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="spacy", lang=nlp, train_size=1)
# <spacy.tokens._serialize.DocBin object at 0x280613af0>

Transformers

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="transformers", train_size=1)
# {'title': 'My title', 'content': 'My content', 'label': 0}

Peft (LoRA)

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="peft", train_size=1)
# {'title': 'My title', 'content': 'My content', 'label': 0}

SpanMarker

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="span_marker", train_size=1)
# {'title': 'My title', 'content': 'My content', 'label': 0}

Spark NLP

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="spark-nlp", train_size=1)
# <pd.DataFrame>

TRL

import argilla as rg

dataset_rg = rg.load("<my_dataset>")
dataset_rg.prepare_for_training(framework="trl", task=..., train_size=1)

CLI 支持#

我们还为 ArgillaTrainer 提供 CLI 支持。例如，在外部计算机上执行训练时可以使用它。请注意，–update-config-kwargs 始终对相应的类使用 update_config() 方法。因此，您应该考虑到这一点，通过传递 JSON 可序列化字符串来通过 CLI 命令配置训练。

Usage: python -m argilla train [OPTIONS] COMMAND [ARGS]...

Starts the ArgillaTrainer.

Options:
--name                        TEXT                                                      The name of the dataset to be used for training. [default: None]
--framework                   [transformers|peft|setfit|spacy|                          The framework to be used for training. [default: None]
                            spacy-transformers|span_marker|spark-nlp|
                            openai|trl|trlx|sentence-transformers]
--workspace                   TEXT                                                      The workspace to be used for training. [default: None]
--limit                       INTEGER                                                   The number of record to be used. [default: None]
--query                       TEXT                                                      The query to be used. [default: None]
--model                       TEXT                                                      The modelname or path to be used for training. [default: None]
--train-size                  FLOAT                                                     The train split to be used. [default: 1.0]
--seed                        INTEGER                                                   The random seed number. [default: 42]
--device                      INTEGER                                                   The GPU id to be used for training. [default: -1]
--output-dir                  TEXT                                                      Output directory for the saved model. [default: model]
--update-config-kwargs        TEXT                                                      update_config() kwargs to be passed as a dictionary. [default: {}]
--help                                                                                  Show this message and exit.

🦾 微调 LLM 和其他语言模型#

反馈数据集#

ArgillaTrainer#

支持的框架#

训练配置#

TrainingTask#

过滤和排序用于训练的数据集#

Huggingface Hub 集成#

模型卡生成#

将您的模型上传到 Huggingface Hub#

任务#

文本分类#

背景#

训练#

问题回答#

背景#

训练#

句子相似度#

背景#

训练#

监督式微调#

背景#

训练#

奖励建模#

背景#

训练#

近端策略优化#

背景#

训练#

直接偏好优化#

背景#

训练#

聊天完成#

背景#

训练#

其他数据集#

ArgillaTrainer#

支持的框架#

训练配置#

任务#

文本分类#

背景#

训练#

Token 分类#

背景#

训练#

Text2Text#

背景#

训练#

其他选项#

准备训练#

CLI 支持#

`ArgillaTrainer`#

`TrainingTask`#

`ArgillaTrainer`#