Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mathstodon.xyz is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Mastodon instance for maths people. We have LaTeX rendering in the web interface!

Administered by:

Server stats:

2.8K
active users

mathstodon.xyz: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#reproduciblecomputing

1 post1 participant0 posts today
Christian Meesters<p>Have been on a little holiday and meanwhile my pull request for the HPC Certification Forum (<a href="https://fediscience.org/tags/HPCCF" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPCCF</span></a> - we just _need_ a hashtag 😉 ) on workflow management systems (that is the skill definitions) has been approved! Thanks to <span class="h-card" translate="no"><a href="https://academiccloud.social/@jkunkel1" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@<span>jkunkel1</span></a></span> !</p><p>To me, it is important that the <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> community realizes the value of automation in data analysis which, in turn, is a great leap for <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a> !</p><p><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenScience</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a></p>
Christian Meesters<p>The <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SnakemakeHackathon2025</span></a> just induced another collaboration for me!</p><p>It is always good to work along with like-minded people, interested in <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a>, <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> and <a href="https://fediscience.org/tags/Bioinformatics" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Bioinformatics</span></a> </p><p><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenScience</span></a></p>
Christian Meesters<p>Today is the day of closed pull request for <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Snakemake</span></a>. The <a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SnakemakeHackathon2025</span></a> participants worked at full speed!</p><p>We decided to write a white-paper summarizing our achievements rather than posting individual things. Suffice to say, that also the documentation made a great leap towards better readability!</p><p><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenScience</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a></p>
Christian Meesters<p><a href="https://fediscience.org/tags/SnakemakeHackathon2025" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SnakemakeHackathon2025</span></a> ! We started!</p><p>At the CERN for better <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a> and <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a> .</p>
Christian Meesters<p>There are many HPC admins who prohibit using considerable CPU time on login nodes. This is understandable.</p><p>I want to take this opportunity to provide a data point. My student has measured the accumulated CPU time (user + system) for a 9 h (precise: 33343 s) run of a Snakemake workflow. It was 225 s or about 0.67 % - including jobs which were carried out on this login node, e.g. `mv`, `ln` or download of data.</p><p>There is certainly room for improvements. There will ever be room for improvements.</p><p>But my dear fellow admins: Running Snakemake on login nodes as a shepherd of jobs, will impair nobodies work.</p><p>Over time, I will certainly gather more and different statistics. And will invest time in necessary improvements. Regarding CPU time for checking job status, however, I believe to have demonstrated, that this is a pretty high hanging fruit.</p><p><a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Snakemake</span></a> <a href="https://fediscience.org/tags/reproduciblecomputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>reproduciblecomputing</span></a></p>
Christian Meesters<p>I will continue to find it disturbing, if new <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> cluster users explicitly instruct a program to use one core/cpu only and then complain that the cluster is so slow. Slower than their basement server.</p><p>Usually they do not spot their mistake on their own.</p><p>But THIS is actually NOT the disturbing part: such users also tend to always use default parameters. This might or might not be the sensible thing to do for their problem. Also, when reading papers, software parameterization frequently is not reported.</p><p>We have a long way to go.</p><p><a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a> <a href="https://fediscience.org/tags/reproduciblecomputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>reproduciblecomputing</span></a></p>
Christian Meesters<p>Again, we have a new patch-level release of the <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Snakemake</span></a> executor for <a href="https://fediscience.org/tags/SLURM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SLURM</span></a> on <a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> systems.</p><p>Turned out, that some clusters do not allow account checking with `sacctmgr` (which was there for historic reasons), hence, we now have a fall-back to `sshare`. </p><p>See <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v0.12.1" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/snakemake/snakemake</span><span class="invisible">-executor-plugin-slurm/releases/tag/v0.12.1</span></a> for details.</p><p><a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenScience</span></a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a></p>
Christian Meesters<p>We have a new release for the <a href="https://fediscience.org/tags/SLURM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SLURM</span></a> support plugin of <a href="https://fediscience.org/tags/Snakemake" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>Snakemake</span></a> !</p><p>It's a minor feature release which enables custom log directories and auto-deletion of successful job logs (such that not zillions of zero-meaning files accumulate!).</p><p>Check it out at: <a href="https://github.com/snakemake/snakemake-executor-plugin-slurm/releases/tag/v0.12.0" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://</span><span class="ellipsis">github.com/snakemake/snakemake</span><span class="invisible">-executor-plugin-slurm/releases/tag/v0.12.0</span></a></p><p>It took a while to get to this release (stress, family, and sickness took a toll). Hopefully, a future release will not take that long to be realized — the feature request list is considerable. 😉 </p><p><a href="https://fediscience.org/tags/HPC" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>HPC</span></a> <a href="https://fediscience.org/tags/ReproducibleResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleResearch</span></a> <a href="https://fediscience.org/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://fediscience.org/tags/OpenScience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>OpenScience</span></a></p>
socallinuxexpo<p>Explore the world of reproducible computing with NixOS in Matthew Croughan's talk at <a href="https://social.linux.pizza/tags/SCaLE21X" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>SCaLE21X</span></a>! Discover how to turn your old phone into a Linux computer, experiment with AI models, and more. <a href="https://social.linux.pizza/tags/NixOS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>NixOS</span></a> <a href="https://social.linux.pizza/tags/ReproducibleComputing" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#<span>ReproducibleComputing</span></a> <a href="https://www.socallinuxexpo.org/scale/21x/presentations/nix-planet" rel="nofollow noopener noreferrer" translate="no" target="_blank"><span class="invisible">https://www.</span><span class="ellipsis">socallinuxexpo.org/scale/21x/p</span><span class="invisible">resentations/nix-planet</span></a></p>
SearchLive feeds
About