Create accountLogin

Recent searches

No recent searches

Search options

Only available when logged in.
mathstodon.xyz is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Mastodon instance for maths people. We have LaTeX rendering in the web interface!

Administered by:

Server stats:

2.9K
active users

mathstodon.xyz: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.7

#arxiv

14 posts13 participants1 post today
Public *
Curated Hacker News

NoProp: Training neural networks without back-propagation or forward-propagation

arxiv.org/abs/2503.24322

arXiv logo
arXiv.orgNoProp: Training Neural Networks without Back-propagation or Forward-propagationThe canonical deep learning approach for learning requires computing a gradient term at each layer by back-propagating the error signal from the output towards each learnable parameter. Given the stacked structure of neural networks, where each layer builds on the representation of the layer below, this approach leads to hierarchical representations. More abstract features live on the top layers of the model, while features on lower layers are expected to be less abstract. In contrast to this, we introduce a new learning method named NoProp, which does not rely on either forward or backwards propagation. Instead, NoProp takes inspiration from diffusion and flow matching methods, where each layer independently learns to denoise a noisy target. We believe this work takes a first step towards introducing a new family of gradient-free learning methods, that does not learn hierarchical representations -- at least not in the usual sense. NoProp needs to fix the representation at each layer beforehand to a noised version of the target, learning a local denoising process that can then be exploited at inference. We demonstrate the effectiveness of our method on MNIST, CIFAR-10, and CIFAR-100 image classification benchmarks. Our results show that NoProp is a viable learning algorithm which achieves superior accuracy, is easier to use and computationally more efficient compared to other existing back-propagation-free methods. By departing from the traditional gradient based learning paradigm, NoProp alters how credit assignment is done within the network, enabling more efficient distributed learning as well as potentially impacting other characteristics of the learning process.
#arxiv
Public
Curated Hacker News

My prediction after GPT-4o image generation

arxiv.org/abs/2504.06256

arXiv logo
arXiv.orgTransfer between Modalities with MetaQueriesUnified multimodal models aim to integrate understanding (text output) and generation (pixel output), but aligning these different modalities within a single architecture often demands complex training recipes and careful data balancing. We introduce MetaQueries, a set of learnable queries that act as an efficient interface between autoregressive multimodal LLMs (MLLMs) and diffusion models. MetaQueries connects the MLLM's latents to the diffusion decoder, enabling knowledge-augmented image generation by leveraging the MLLM's deep understanding and reasoning capabilities. Our method simplifies training, requiring only paired image-caption data and standard diffusion objectives. Notably, this transfer is effective even when the MLLM backbone remains frozen, thereby preserving its state-of-the-art multimodal understanding capabilities while achieving strong generative performance. Additionally, our method is flexible and can be easily instruction-tuned for advanced applications such as image editing and subject-driven generation.
#arxiv
Public *
Michael Busch

Today on the #arXiv :

Lau et al. 2025, "Revealing a main-sequence star that consumed a planet with JWST" - arxiv.org/pdf/2504.07275

Sometimes, planets fall onto stars.

ZTF SLRN-2020 was first reported by De et al. 2023, found in survey data from Palomar: nature.com/articles/s41586-023 .

And compare to Vissapragada et al. 2022 about a planet that has not quite yet fallen onto another star: iopscience.iop.org/article/10.

Public
Daniel Fischer

Wanna have the Universe talk to you? Now on offer by #ArXiv where AI-generated summaries of new high energy astrophysics papers can be listened to: blog.arxiv.org/2025/04/08/arxi - like e.g. arxiv.org/abs/2504.07051 (Revisiting X-ray polarization of the shell of Cassiopeia A using spectropolarimetric analysis) which then sounds like sciencecast.org/casts/4gohkbij

blog.arxiv.orgarXiv Pilots Audio Summaries in Partnership with ScienceCast  – arXiv blog
Public
Curated Hacker News

ProtoGS: Efficient and High-Quality Rendering with 3D Gaussian Prototypes

arxiv.org/abs/2503.17486

arXiv logo
arXiv.orgProtoGS: Efficient and High-Quality Rendering with 3D Gaussian Prototypes3D Gaussian Splatting (3DGS) has made significant strides in novel view synthesis but is limited by the substantial number of Gaussian primitives required, posing challenges for deployment on lightweight devices. Recent methods address this issue by compressing the storage size of densified Gaussians, yet fail to preserve rendering quality and efficiency. To overcome these limitations, we propose ProtoGS to learn Gaussian prototypes to represent Gaussian primitives, significantly reducing the total Gaussian amount without sacrificing visual quality. Our method directly uses Gaussian prototypes to enable efficient rendering and leverage the resulting reconstruction loss to guide prototype learning. To further optimize memory efficiency during training, we incorporate structure-from-motion (SfM) points as anchor points to group Gaussian primitives. Gaussian prototypes are derived within each group by clustering of K-means, and both the anchor points and the prototypes are optimized jointly. Our experiments on real-world and synthetic datasets prove that we outperform existing methods, achieving a substantial reduction in the number of Gaussians, and enabling high rendering speed while maintaining or even enhancing rendering fidelity.
#arxiv
Public
Curated Hacker News

NNN: Next-Generation Neural Networks for Marketing Mix Modeling

arxiv.org/abs/2504.06212

arXiv logo
arXiv.orgNNN: Next-Generation Neural Networks for Marketing Mix ModelingWe present NNN, a Transformer-based neural network approach to Marketing Mix Modeling (MMM) designed to address key limitations of traditional methods. Unlike conventional MMMs which rely on scalar inputs and parametric decay functions, NNN uses rich embeddings to capture both quantitative and qualitative aspects of marketing and organic channels (e.g., search queries, ad creatives). This, combined with its attention mechanism, enables NNN to model complex interactions, capture long-term effects, and potentially improve sales attribution accuracy. We show that L1 regularization permits the use of such expressive models in typical data-constrained settings. Evaluating NNN on simulated and real-world data demonstrates its efficacy, particularly through considerable improvement in predictive power. Beyond attribution, NNN provides valuable, complementary insights through model probing, such as evaluating keyword or creative effectiveness, enhancing model interpretability.
#arxiv
Public
MottG

The Harvard ADS search engine
has many advanced features if you
select "Modern Form" at the top of the screen to get the Bumblebee search
interface. You can create your own virtual library of publications
and make it public. You can search for citations that cite a collection of papers. There is a "Citation Helper" facility that may help you identify missing papers.

ui.adsabs.harvard.edu/

There is online help:
ui.adsabs.harvard.edu/help/

Screenshot of user interface with search facets for Harvard ADS Bumblebee user interface.
#research#astronomy#science
Public
Curated Hacker News

Can reinforcement learning for LLMs scale beyond math and coding tasks? Probably

arxiv.org/abs/2503.23829

arXiv logo
arXiv.orgExpanding RL with Verifiable Rewards Across Diverse DomainsReinforcement learning (RL) with verifiable rewards (RLVR) has shown promising results in mathematical reasoning and coding tasks where well-structured reference answers are available. However, its applicability to broader domains remains underexplored. In this work, we study the extension of RLVR to more diverse domains such as medicine, chemistry, psychology, and economics. We observe high agreement in binary judgments across different large language models (LLMs) when objective reference answers exist, which challenges the necessity of large-scale annotation for training domain-specific reward models. To address the limitations of binary rewards when handling unstructured reference answers, we further incorporate model-based soft scoring into RLVR to improve its flexibility. Our experiments show that a distilled generative reward model can serve as an effective cross-domain verifier, providing reliable reward signals for RL without requiring domain-specific annotations. By fine-tuning a base 7B model using various RL algorithms against our reward model, we obtain policies that outperform state-of-the-art open-source aligned LLMs such as Qwen2.5-72B-Instruct and DeepSeek-R1-Distill-Qwen-32B by a large margin, across domains in free-form answer settings. This also strengthens RLVR's robustness and scalability, highlighting its potential for real-world applications with noisy or weak labels.
#arxiv#llm#llms
Public
Curated Hacker News

Beyond Quacking: Deep Integration of Language Models and RAG into DuckDB

arxiv.org/abs/2504.01157

arXiv logo
arXiv.orgBeyond Quacking: Deep Integration of Language Models and RAG into DuckDBKnowledge-intensive analytical applications retrieve context from both structured tabular data and unstructured, text-free documents for effective decision-making. Large language models (LLMs) have made it significantly easier to prototype such retrieval and reasoning data pipelines. However, implementing these pipelines efficiently still demands significant effort and has several challenges. This often involves orchestrating heterogeneous data systems, managing data movement, and handling low-level implementation details, e.g., LLM context management. To address these challenges, we introduce FlockMTL: an extension for DBMSs that deeply integrates LLM capabilities and retrieval-augmented generation (RAG). FlockMTL includes model-driven scalar and aggregate functions, enabling chained predictions through tuple-level mappings and reductions. Drawing inspiration from the relational model, FlockMTL incorporates: (i) cost-based optimizations, which seamlessly apply techniques such as batching and caching; and (ii) resource independence, enabled through novel SQL DDL abstractions: PROMPT and MODEL, introduced as first-class schema objects alongside TABLE. FlockMTL streamlines the development of knowledge-intensive analytical applications, and its optimizations ease the implementation burden.
#arxiv#duckdb
Public *
もく

[2504.01830] Is Lorentz invariance violation found?

arxiv.org/abs/2504.01830

> ...Very recently, the Carpet collaboration has completed the full data analysis, reporting further support for their previously detected photon now at ${\cal E} = 300^{+ 43}_{- 38} \, {\rm TeV}$, which manifestly clashes with conventional physics
...
> If confirmed by future observations our finding would represent the first positive result in quantum gravity phenomenology.

@physics
#Physics #Relativity #arXiv

Figure 1: Photon survival probability P⁢(ℰ;γ→γ) versus energy ℰ taking into account ALP and LIV effects separately (upper and central panels) and together (lower panel). Conventional physics is reported in all panels for comparison.
Figure 2: Observed SED versus energy ℰ taking into account ALP and LIV effects separately (upper and central panels) and together (lower panel). Observed SED versus energy ℰ in conventional physics is reported in all panels for comparison. In all panels the black dotted line is the intrinsic SED of GRB 221009A as measured by LHAASO lhaaso1 ; lhaaso2 and extended up to the Carpet energies carpet3 .
Public
Curated Hacker News

DeepSeek: Inference-Time Scaling for Generalist Reward Modeling

arxiv.org/abs/2504.02495

arXiv logo
arXiv.orgInference-Time Scaling for Generalist Reward ModelingReinforcement learning (RL) has been widely adopted in post-training for large language models (LLMs) at scale. Recently, the incentivization of reasoning capabilities in LLMs from RL indicates that $\textit{proper learning methods could enable effective inference-time scalability}$. A key challenge of RL is to obtain accurate reward signals for LLMs in various domains beyond verifiable questions or artificial rules. In this work, we investigate how to improve reward modeling (RM) with more inference compute for general queries, i.e. the $\textbf{inference-time scalability of generalist RM}$, and further, how to improve the effectiveness of performance-compute scaling with proper learning methods. For the RM approach, we adopt pointwise generative reward modeling (GRM) to enable flexibility for different input types and potential for inference-time scaling. For the learning method, we propose Self-Principled Critique Tuning (SPCT) to foster scalable reward generation behaviors in GRMs through online RL, to generate principles adaptively and critiques accurately, resulting in $\textbf{DeepSeek-GRM}$ models. Furthermore, for effective inference-time scaling, we use parallel sampling to expand compute usage, and introduce a meta RM to guide voting process for better scaling performance. Empirically, we show that SPCT significantly improves the quality and scalability of GRMs, outperforming existing methods and models in various RM benchmarks without severe biases, and could achieve better performance compared to training-time scaling. DeepSeek-GRM still meets challenges in some tasks, which we believe can be addressed by future efforts in generalist reward systems. The models will be released and open-sourced.
#arxiv
Public
Curated Hacker News

Banked Memories for Soft SIMT Processors

arxiv.org/abs/2503.24132

arXiv logo
arXiv.orgBanked Memories for Soft SIMT ProcessorsRecent advances in soft GPGPU architectures have shown that a small (<10K LUT), high performance (770 MHz) processor is possible in modern FPGAs. In this paper we architect and evaluate soft SIMT processor banked memories, which can support high bandwidth (up to 16 ports) while maintaining high speed (over 770 MHz). We compare 9 different memory architectures, including simpler multi-port memories, and run a total of 51 benchmarks (different combinations of algorithms, data sizes and processor memories) to develop a comprehensive set of data which will guide the reader in making an informed memory architecture decision for their application. Our benchmarks are comprised of matrix transpositions (memory intensive) and FFTs (split between memory accesses, floating point, and integer computations) to provide a balanced evaluation. We show that the simpler (but more memory block intensive) multi-port memories offer higher performance than the more architecturally complex banked memories for many applications, especially for smaller memories, but the effective footprint cost of the multi-port memories quickly becomes prohibitive as dataset sizes increase. Our banked memory implementation results - high bandwidth, high Fmax, and high density - can be used for other FPGA applications as well, such as HLS (High Level Synthesis).
#arxiv
SearchLive feeds
About