There's no need to be shy about asking: why is #CSV better?
Our new course ‘Quality and Consistent Data with #OpenDataEditor’ takes you by the hand.
In this lesson, you learn the anatomy of CSV and why it's a key format for #opendata.
Recent searches
Search options
#csv
3 posts2 participants1 post today
"Excel hates CSV - It clearly means CSV must be doing something right"
-- xan, the #CSV magician
https://github.com/medialab/xan/blob/master/docs/LOVE_LETTER.md
GitHubxan/docs/LOVE_LETTER.md at master · medialab/xanThe CSV magician. Contribute to medialab/xan development by creating an account on GitHub.
Parsing CSV with units in the header · Issue #166 · hgrecco/pint-pandas
https://github.com/hgrecco/pint-pandas/issues/166
Now we can read a #csv file with a header like `time / s,mass / g` into #pandas and call `.pint.quantify()` to get a #dataframe in which the columns have #units as in #Pints !
Handy for CSV restricted to single-row headers, as in Confluence Databases and Microsoft Lists.
#TIL about #Visidata: https://jsvine.github.io/intro-to-visidata/
If you ever need to wrangle, filter, sort, edit, navigate, subset or get basic statistics out of #data in #CSV format (comma-separated values), this is a super useful piece of software. It takes raw data and implements a very handy subset of functionality from spreadsheets directly on top, with both a keyboard (modal Vi-style operation) and menu/mouse driven operation.
Nice one to have in the toolchest.
jsvine.github.ioAn Introduction to VisiData — An Introduction to VisiData
XAN: A Modern CSV-Centric Data Manipulation Toolkit for the Terminal
GitHubGitHub - medialab/xan at console.devThe CSV magician. Contribute to medialab/xan development by creating an account on GitHub.
GitHubGitHub - medialab/xan at console.devThe CSV magician. Contribute to medialab/xan development by creating an account on GitHub.
Not sure who needs this (apart from me, of course), but just pushed my rof2csv tool:
https://github.com/branch-predictor/rof2csv
Use it to convert Rigol's ROF files (produced by Rigol DP800 series programmable power supplies) into CSV files that can be fed into Excel or LibreOffice Calc.
#programming #SoftwareDevelopment #rigol #rof #csv
GitHubGitHub - branch-predictor/rof2csv: rof2csv - command-line tool to convert Rigol ROF files to CSVrof2csv - command-line tool to convert Rigol ROF files to CSV - branch-predictor/rof2csv
Looking forward to #DFD2025. What #FLOSS tools do you use to read #CSV files? Any FOSS #ETL tools to copy CSVs to/from databases? Is there a #Free, #OpenSource ODBC driver for connecting to CSV files?
#AskFedi #AskFediverse #DFF
Generate seeders from existing table data with orangehill/iseed
Don't bury your research data in #PDF files. https://www.linkedin.com/posts/jonippolito_archives-collections-data-activity-7305940793683267584-_aHF
Put raw data in data repositories and use open formats to save them and make them reusable. #CSV / TSV are the perfect format for tabulated data.
www.linkedin.com#archives #collections #data #digitalcuration #digitization… | Jon IppolitoCan AI save PDF from itself? Probably not, so stop using PDF to archive your documents.
PDFs are not easily accessible, hide their dependencies and security vulnerabilities, and rely on a 700-page spec developed by a proprietary vendor rather than an open standards body.
Most importantly, they're a bitch to extract data from. (Have you tried copy-pasting text from a two-column PDF?) Along with bitmapped PDFs that are basically images, there are even searchable PDFs whose creators have accidentally or deliberately mixed up the letters in the font table to make them non-copiable. These failure points are why Arxiv started giving readers the option to read papers in HTML.
This sounds like a good job for AI, and in fact Mistral just launched an API for converting PDF to the simple, readable Markdown format. Unfortunately, independent tests have showed poor results https://lnkd.in/eyktiSaa. When you upload a PDF as background for your Custom GPT or NotebookLM podcast, this poor accessibility just compounds potential errors at inference time.
But it's a lot worse when a PDF is consulted in a life-or-death situation. Optical LLMs have been caught mismatching column headings and lines of data. Imagine a doctor who uses AI to consult a PDF table of drug protocols and prescribes 500mg of a potentially toxic chemotherapy agent instead of 50mg.
Especially in science, the value of information is the ability to re-use it. Yet archivists in the digital era have spent way too much time figuring out how to get data into a repository compared to getting it out. As I've said before, a collection of PDFs isn't an archive; it's a tomb.
#Archives #Collections #Data #DigitalCuration #Digitization #DigitalPreservation #AIethics #GenerativeAI #GenAI #LLM #MachineLearning #OCR
Dans les coulisses du fonds #ÉricRouleau : faciliter la #rétroconversion dans #Calames avec #Python #Archives #MMSH
https://phonotheque.hypotheses.org/49828
Un billet qui décrit précisément l'écriture d'un bout de #code pour passer d'un #CSV à des notices #EAD, avec en prime les infos sur un type incroyable Éric Rouleau
Archives de la recherche & PhonothèqueDans les coulisses du fonds Éric Rouleau : faciliter la rétroconversion dans Calames avec PythonOuvrir le fonds Éric Rouleau, c’est voir une grande partie de l’histoire du XXe siècle se dessiner sous nos yeux, à travers le regard d’un témoin privilégié qui a été journaliste, puis ambassadeur, au contact des hommes politiques et des dirigeants d’Europe et du Moyen-Orient. En 1943, à l’âge de 17 ans, Élie Raffoul (qui … Continuer la lecture de Dans les coulisses du fonds Éric Rouleau : faciliter la rétroconversion dans Calames avec Python →
Comparing #CSV data files, are there any great (offline) tools? Ideally somehow visual, like there are plenty of nice visual interfaces for "diff" output for source code. #analytics #dataQuality
[
Utiliser des données en #opendata n°6] Vous avez téléchargé des données et souhaitez ouvrir le fichier.
L'équipe de data.gouv.fr propose un rappel de quelques options possibles pour les formats #CSV, #JSON, #XML, #GeoJSON, #shapefile et #KML dans son guide : https://guides.data.gouv.fr/reutiliser-des-donnees/guide-traitement-et-analyse-de-donnees/manipuler-des-donnees/ouvrir-des-donnees
Continued thread
• 
Customizable configuration with rules, allowlists, and entropy checks to reduce false positives
• 
Flexible reporting in multiple formats (#JSON, #CSV, #JUnit, #SARIF) with custom template options
https://github.com/gitleaks/gitleaks
GitHubGitHub - gitleaks/gitleaks: Find secrets with Gitleaks 🔑Find secrets with Gitleaks 🔑. Contribute to gitleaks/gitleaks development by creating an account on GitHub.
The oldschool web/tech side of me is smiling.
"A love letter to the #CSV format"
https://github.com/medialab/xan/blob/master/docs/LOVE_LETTER.md
"Or why people pretending CSV is dead are wrong... #Excel hates CSV which clearly means CSV must be doing something right."
GitHubxan/docs/LOVE_LETTER.md at master · medialab/xanThe CSV magician. Contribute to medialab/xan development by creating an account on GitHub.
League/csv 9.22.0 is out with a brand new class that seats between the Reader and the Writer called the Buffer. It allows buffering you CSV document and performing some CRUD operation on the data before storing it. You can do a lot more with it check the documentation https://csv.thephpleague.com/9.0/writer/buffer/
Helped my wife to simplify her work life a bit. She was frustrated with how many clicks it took her to log her work hours in #bamboohr. With the help of some coding AI she now extracts tagged events from her Google calendar into a #csv. Then she uses a little #python script to talk to the Bamboo API to add the hours. The AI generated code needed a bit of tweaking but she was amazed how easy you can get a working prototype.
Now it saves her a lot of time manually logging her hours.
