detector_benchmark.detector =========================== .. py:module:: detector_benchmark.detector Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/detector_benchmark/detector/bert_detector/index /autoapi/detector_benchmark/detector/detector/index /autoapi/detector_benchmark/detector/detector_loader/index /autoapi/detector_benchmark/detector/fast_detect_gpt/index /autoapi/detector_benchmark/detector/gpt_zero_detector/index /autoapi/detector_benchmark/detector/watermark_detector/index Classes ------- .. autoapisummary:: detector_benchmark.detector.BertDetector detector_benchmark.detector.Detector detector_benchmark.detector.FastDetectGPT detector_benchmark.detector.WatermarkDetector detector_benchmark.detector.DetectorLoader detector_benchmark.detector.GPTZero Package Contents ---------------- .. py:class:: BertDetector(model: torch.nn.Module, tokenizer: transformers.PreTrainedTokenizerBase, device: str) Bases: :py:obj:`detector_benchmark.detector.detector.Detector` .. py:attribute:: model .. py:attribute:: tokenizer .. py:attribute:: device .. py:method:: detect(texts: list[str], batch_size: int, detection_threshold: float = 0.0) -> tuple[list[int], list[float], list[int]] Detect the if the texts given as input are AI-generated (label 1) or human-written (label 0). Returns the predicted lables with argmax, the logits of the positive class and the predicted labels with the given detection threshold instead of the argmax. Parameters: ---------- texts: list[str] The texts to detect batch_size: int The batch size to use for detection detection_threshold: float The threshold to use for detection. Default is 0.0. Returns: ---------- tuple[list[int], list[float], list[int]] The predicted labels with argmax, the logits of the positive class and the predicted labels with the given detection threshold instead of the argmax. .. py:class:: Detector Bases: :py:obj:`abc.ABC` Helper class that provides a standard way to create an ABC using inheritance. .. py:method:: detect(texts: list[str], batch_size: int, detection_threshold: float = 0.0) -> tuple[list[int], list[float], list[int]] :abstractmethod: Detect the watermark in the texts. Parameters: ---------- texts: list[str] The texts to detect the watermark in batch_size: int The batch size detection_threshold: float The threshold to use for the detection Returns: ------- tuple[list[int], list[float], list[int]] The predictions, the logits, and the predictions at the threshold .. py:class:: FastDetectGPT(ref_model, scoring_model, ref_tokenizer, scoring_tokenizer, device) Bases: :py:obj:`detector_benchmark.detector.detector.Detector` .. py:attribute:: ref_model .. py:attribute:: scoring_model .. py:attribute:: ref_tokenizer .. py:attribute:: scoring_tokenizer .. py:attribute:: device .. py:method:: get_samples(logits, labels) -> torch.Tensor Get the samples from the logits. Parameters: ---------- logits: torch.Tensor The logits labels: torch.Tensor The labels .. py:method:: get_likelihood(logits, labels) -> torch.Tensor Get the likelihood from the logits. Parameters: ---------- logits: torch.Tensor The logits labels: torch.Tensor The labels .. py:method:: get_sampling_discrepancy(logits_ref, logits_score, labels) -> torch.Tensor Get the sampling discrepancy from the logits. Parameters: ---------- logits_ref: torch.Tensor The logits of the reference model logits_score: torch.Tensor The logits of the scoring model labels: torch.Tensor The labels .. py:method:: get_sampling_discrepancy_analytic(logits_ref, logits_score, labels) -> torch.Tensor Get the sampling discrepancy from the logits. Parameters: ---------- logits_ref: torch.Tensor The logits of the reference model logits_score: torch.Tensor The logits of the scoring model .. py:class:: ProbEstimatorFastDetectGPT(args=None, ref_path=None) Probability estimator for the FastDetectGPT detector. .. py:attribute:: real_crits :value: [] .. py:attribute:: fake_crits :value: [] .. py:method:: crit_to_prob(crit) -> float Convert the criterion to probability. Parameters: ---------- crit: float The criterion Returns: ---------- float The probability .. py:method:: detect(texts: list[str], batch_size: int, detection_threshold: float = 0.5) -> tuple[list[int], list[float], list[int]] Detect the watermark in the texts. Parameters: ---------- texts: list[str] The texts to detect the watermark in batch_size: int The batch size detection_threshold: float The detection threshold Returns: ---------- tuple[list[int], list[float], list[int]] The predictions, the probabilities and the predictions at the threshold .. py:class:: WatermarkDetector(watermarking_scheme: detector_benchmark.watermark.auto_watermark.AutoWatermark, detection_threshold: float) Bases: :py:obj:`detector_benchmark.detector.detector.Detector` .. py:attribute:: watermarking_scheme .. py:attribute:: detection_threshold .. py:method:: detect(texts: list[str], batch_size: int, detection_threshold: float) -> tuple[list[int], list[float], list[int]] Detect the if the texts given as input are watermarked (label 1) or not (label 0). Parameters: ---------- texts: list[str] The texts to detect batch_size: int The batch size detection_threshold: float The threshold to use for the detection Returns: ---------- tuple[list[int], list[float], list[int]] The predictions, the logits for the positive class, and the predictions at the threshold .. py:class:: DetectorLoader(cfg: dict, detector_name: str, device: str, weights_checkpoint: str = None, local_weights: bool = False) .. py:attribute:: cfg .. py:attribute:: detector_name .. py:attribute:: device .. py:attribute:: weights_checkpoint .. py:attribute:: local_weights .. py:method:: load() -> detector_benchmark.detector.detector.Detector Load the detector based on the given configuration (init). Returns: ---------- Detector The loaded detector .. py:class:: GPTZero(api_key, debug_mode=False) Bases: :py:obj:`detector_benchmark.detector.detector.Detector` .. py:attribute:: api_key .. py:attribute:: debug_mode .. py:method:: predict_gpt_zero(text, api_key, debug_mode=False) -> dict Predict the GPT-Zero score for the text. Parameters: ---------- text: str The text to predict api_key: str The API key debug_mode: bool Whether to print the debug information Returns: ---------- dict The prediction result .. py:method:: detect(texts: list, batch_size: int, detection_threshold: float = 0.5) -> tuple[list[int], list[float], list[int]] Detect the GPT-Zero score for the texts. Parameters: ---------- texts: list The texts to detect batch_size: int The batch size detection_threshold: float The threshold to use for the detection Returns: ---------- tuple[list[int], list[float], list[int]] The predictions, the logits for the positive class, and the predictions at the threshold