clinicalbert-a12748a1·3 events·first seen Aliases: ClinicalBERT
Researchers from CaresAI evaluated biomedical transformer models (ClinicalBERT, PubMedBERT, BioBERT, MedCPT) for detecting dosing errors in clinical trial protocols, combining text embeddings with structured metadata and classical ML classifiers. BioBERT achieved the best single-encoder performance at ROC-AUC 0.794, while gradient boosting and SVM ensembles reached 0.821–0.853. The study finds that domain alignment of the encoder matters more than stacking multiple embeddings, and demonstrates a practical NLP pipeline for clinical trial safety monitoring.
Researchers present a systematic audit of representational bias in ClinicalBERT, a BERT-based model pretrained on MIMIC-III clinical discharge summaries, using two probing methodologies: Log Probability Bias Analysis and Masked Language Model probing across 98 clinical sentence templates and eight intersectional race-gender combinations. Of 32 statistically significant findings, 65.6% contradict observed corpus distributions, rising to 80% for Black patients and 87.5% for agency attribution under MLM probing. The key finding is that bias in ClinicalBERT operates predominantly through model-internal amplification rather than simple inheritance from training data, which has direct implications for clinical AI safety and deployment. This challenges the assumption that auditing training corpora is sufficient to characterize model bias.
Researchers conduct a sensitivity analysis of both general-purpose and medical-specific LLMs using the MedMCQA benchmark, testing robustness to lexical and syntactic prompt perturbations. The study finds that even minor phrasing changes can alter clinical advice, and adversarial prompts can produce dangerous outputs such as incorrect dosages or omitted critical findings. Both general-purpose models (GPT-3.5, Llama 3) and domain-specific models (ClinicalBERT, BioLlama3, BioBERT) exhibit this fragility, with syntactic reordering and misleading contextual cues proving more destabilizing than simple paraphrasing.