Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mathstodon.xyz is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Mastodon instance for maths people. We have LaTeX rendering in the web interface!

Administered by:

Server stats:

2.7K
active users

mathstodon.xyz: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#textanalysis

0 posts0 participants0 posts today
Public *
Mario Angst

🇪🇺 Want to analyze text from the EU public consultations? EU public consultations are a way in which the EU invites the broader public to publicly comment on upcoming legislation.

📦 :python: I just published a first version of a Python package {eu-consultations} to scrape and extract text from the EU website:
github.com/marioangst/eu_consu

- download consultation data as displayed on the EU's frontend into a validated form
- download associated files (this is the hard part about analysing this data - lots of feedback is in .docx and .pdf files)
- extract text from the files using docling and attach to feedback

You get all data in validated form and possibly stored in huge (sorry for that) JSON files ;).

This package is part of an analysis project on feedback the EU has received via the public consultation process on digital policy we plan to present later this year, but I thought let's make some of the tools we use open source way earlier already.

eu-consultations: A Python package for scraping textual data from EU public consultations - marioangst/eu_consultations
GitHubGitHub - marioangst/eu_consultations: eu-consultations: A Python package for scraping textual data from EU public consultationseu-consultations: A Python package for scraping textual data from EU public consultations - marioangst/eu_consultations
#python#textanalysis#policyanalysis
Public
George Macgregor

Useful contribution to discussions in this area, for sure! The results highlight "whether an automated approach that would still require micromanaging and adjusting several variables by the human researcher would, in fact, be more efficient an approach compared to the same tasks performed manually by human labour"

Out of Context! Managing the Limitations of Context Windows in #ChatGPT-4o Text Analyses doi.org/10.46298/jdmdh.15090 #DigitalHumanities #TextAnalysis #LLM #ArtificialIntelligence #GLAMR

EpisciencesOut of Context! Managing the Limitations of Context Windows in ChatGPT-4o Text AnalysesIn recent years, large language model (LLM) applications have surged in popularity, and academia has followed suit. Researchers frequently seek to automate text annotation - often a tedious task – and, to some extent, text analysis. Notably, popular LLMs such as ChatGPT have been studied as both research assistants and analysis tools, revealing several concerns regarding transparency and the nature of AI-generated content. This study assesses ChatGPT’s usability and reliability for text analysis – specifically keyword extraction and topic classification – within an “out-of-the-box” zero-shot or few-shot context, emphasizing how the size of the context window and varied text types influence the resulting analyses. Our findings indicate that text type and the order in which texts are presented both significantly affect ChatGPT’s analysis. At the same time, context-building tends to be less problematic when analyzing similar texts. However, lengthy texts and documents pose serious challenges: once the context window is exceeded, “hallucinated” results often emerge. While some of these issues stem from the core functioning of LLMs, some can be mitigated through transparent research planning.
Public
khushnuma

Mastering these core NLP techniques is crucial for any data scientist dealing with text data. From tokenization to language modeling, each method serves a unique purpose in processing, analyzing, and extracting valuable insights from textual information.

#NLP #DataScience #Tokenization #LanguageModeling #TextAnalysis #TextMining #MachineLearning

read more: blogulr.com/khushnuma7861/topn

Public
Nick Byrd, Ph.D.

Like we found in “Your Health vs. My Liberty” (doi.org/10.1016/j.cognition.20) Yael Rozenblum et al. found that compliance with #publicHealth guidance correlated with indicators of the perceived threat of a viral pandemic.

Also, relying on #misinformation correlated with reliance on simple (vs. complex) #reasoning.

The free paper: doi.org/10.1002/tea.21975

Measures of perceived threat (“motivation”) and compliance (“stance”).
How perceived threat (“motivation”) predicted compliance (“stance”).
Categorization of simple and complex reasoning (with some examples).
How reliance on misinformation correlated with complexity of reasoning and education.
#medicine#health#education
Public
Daniela Schneider

Have you ever wanted to use a #LLM as one step in a workflow?

We integrated #GPT into the open-source analysis platform #useGalaxy, where you can link GPT to several thousand other tools, add more attachments for analysis and make your research reproducible.

galaxyproject.org/news/2024-09

In our example, we uploaded an audio file and used #Whisper to convert it into text, cut out the moderation, and prompted chatGPT to translate it into German.

#DH #textanalysis #tools
@galaxyfreiburg

galaxyproject.orgUsing Large Language Models in complex workflowsUse ChatGPT in your analysis on the Galaxy Server to leverage the Large Language Model in your automated workflows
Public
Fabio Giglietto

📚🇮🇹 New working paper: "Evaluating Embedding Models for Clustering Italian Political News"

This study compares embedding models for unsupervised clustering of Italian political news shared on Facebook before the 2018 and 2022 elections, aiming to advance NLP methods for political text analysis in non-English languages.

Paper: osf.io/preprints/osf/2j9ed

Code & data: github.com/fabiogiglietto/Sema

Feedback welcome!

osf.ioOSF
#NLP#PoliticalScience#TextAnalysis
Public
Jason Robison

Just launched: pycpidr 🎉
github.com/jrrobison1/pycpidr

Python library to determine the propositional idea density of an English text automatically.

Idea density is a measure of the amount of information conveyed relative to the number of words used. This metric has applications in various fields, including linguistics, cognitive science, and healthcare research.
#Python #Linguistics #psychometrics #NLP #TextAnalysis #OpenSource

GitHubGitHub - jrrobison1/pycpidr: Python library to determine the propositional idea density of an English text automatically.Python library to determine the propositional idea density of an English text automatically. - jrrobison1/pycpidr
Public
Axel Pichler

📣 Attention Linguistics & Digital Humanities students! 🎓📚
Join @janispagel and me for the »Prompting, Evaluation, Interpretation: An Introduction to LLMs in Text Analysis« course at the upcoming Deep Learning for Language Analysis Summer School in Cologne: ml-school.uni-koeln.de! 📝🔍
🗓️ Don't miss out – registration is open until June 16th! 🙌
#LLMs #TextAnalysis #NLP #AI #Linguistics #DigitalHumanities #CRETA

ml-school.uni-koeln.dehttp://ml-school.uni-koeln.de | CA3
Public
🧿🪬🍄🌈🎮💻🚲🥓🎃💀🏴🛻🇺🇸

How would you go about creating a filter that blocks posts about things that people hate?

I've thought I could build a text classifier, but it could be hard to train since I'd need to guess whether or not the author hates the thing they are posting about.

I wouldn't want it to become a filter for all current events news, but I suspect that's what it would become.

#fediverse#mastodon#machineLearning
SearchLive feeds
About