Mathstodon

0 posts0 participants0 posts today

Felix Palmen :freebsd: :c64:Still working on <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a>, and currently very busy with improving quality, most of the actual work done inside my <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a> library.After finally supporting <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> and <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a>, I now integrated <a href="https://mastodon.bsd.cafe/tags/xxhash" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#xxhash</a> to completely replace my previous stupid and naive hashing. I also added a more involved <a href="https://mastodon.bsd.cafe/tags/dictionary" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#dictionary</a> class as an alternative to the already existing <a href="https://mastodon.bsd.cafe/tags/hashtable" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#hashtable</a>. While the hashtable's size must be pre-configured and collissions are only ever resolved by storing linked lists, the new dictionary dynamically nests multiple hashtables (using different bits of a single hash value). I hope to achieve acceptable scaling while maintaining also acceptable memory overhead that way ...<a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> already uses both container classes as appropriate.Next I'll probably revisit poser's <a href="https://mastodon.bsd.cafe/tags/threadpool" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#threadpool</a>. I think I could replace <a href="https://mastodon.bsd.cafe/tags/pthread" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pthread</a> condition variables by "simple" <a href="https://mastodon.bsd.cafe/tags/semaphores" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#semaphores</a>, which should also reduce overhead ... <a href="https://github.com/Zirias/swad" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://github.com/Zirias/swad</a><a href="https://mastodon.bsd.cafe/tags/c" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#c</a> <a href="https://mastodon.bsd.cafe/tags/coding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#coding</a>

Felix Palmen :freebsd: :c64:<a href="https://mastodon.bsd.cafe/@pertho" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@pertho</a> The only little drawback compared to epoll is the lack of atomic signal mask setting, so you need a bit more code and a thoughtful structure to handle signals in the same loop. Apart from that, indeed much better than <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a>.Unfortunately, it's not beter than inotify (for a completely different purpose, <a href="https://mastodon.bsd.cafe/tags/file" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#file</a> <a href="https://mastodon.bsd.cafe/tags/monitoring" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#monitoring</a> ... kqueue covers them all). With <a href="https://mastodon.bsd.cafe/tags/inotify" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#inotify</a>, you can for example set a <a href="https://mastodon.bsd.cafe/tags/watch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#watch</a> by path, while <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> requires opened file descriptors. 😞

Felix Palmen :freebsd: :c64:First change since <a href="https://mastodon.bsd.cafe/tags/swad" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#swad</a> 0.2 will actually be a (huge?) improvement to my <a href="https://mastodon.bsd.cafe/tags/poser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poser</a> lib. So far, it was hardwired to use the good old <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a> <a href="https://mastodon.bsd.cafe/tags/select" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#select</a> call. This is perfectly fine for handling around up to 100 (or at least less than 1000, YMMV) clients.Some <a href="https://mastodon.bsd.cafe/tags/select" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#select</a> implementations offer defining the upper limit for checked file descriptors. Added support for that.POSIX also specifies <a href="https://mastodon.bsd.cafe/tags/poll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poll</a>, which has very similar <a href="https://mastodon.bsd.cafe/tags/scalability" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#scalability</a> issues, but slightly different. Added support for this as well.And then, I went on to add support for the <a href="https://mastodon.bsd.cafe/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a>-specific <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> and <a href="https://mastodon.bsd.cafe/tags/BSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BSD</a>-specific <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> (<a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a>, <a href="https://mastodon.bsd.cafe/tags/NetBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NetBSD</a>, <a href="https://mastodon.bsd.cafe/tags/OpenBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenBSD</a>, ...) which are both designed to *solve* any scalability issues 🥳 A little thing that slightly annoyed me about kqueue was that there's no support for temporarily changing the signal mask, so I had to do the silly dance shown in the screenshot. OTOH, it offers changing event filters and getting events in a single call, which I might try to even further optimize ... 😎<a href="https://mastodon.bsd.cafe/tags/C" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#C</a> <a href="https://mastodon.bsd.cafe/tags/coding" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#coding</a>

Несерьёзный ВыдумщикК вопросу использования <a class="hashtag" href="https://idealists.su/tag/epoll" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> вместо хорошо знакомых и «традиционных» select & poll. Т.е. асинхронной работы с чем-либо посредством polling’а и мультиплексирования.Недавно пришлось заниматься реализацией очереди событий для AMQP-CPP. В одном из продуктов решено сделать связь агентских частей с основным «контроллером» через <a class="hashtag" href="https://idealists.su/tag/amqp" rel="nofollow noopener noreferrer" target="_blank">#AMQP</a>, в качестве брокера <a class="hashtag" href="https://idealists.su/tag/rabbitmq" rel="nofollow noopener noreferrer" target="_blank">#RabbitMQ</a> (всё стандартно, обычный кластер и TLS-соединения).Вот только агенты продукта активно используют асинхронно-реактивное программирование с хорошей «горизонтальной масштабируемостью». Когда достигнуто полноценное sharing nothing, не просто горизонтальная масштабируемость через lock-free или wait-free и закон Амдала. Исключается много всего и сразу, как старый-добрый cache ping-pong, так и печаль с false sharing.Отсюда внутри агентов и своё управление потоками с выделениями памяти. Не только в плане heap (динамической памяти, со своими аллокаторами а-ля <a class="hashtag" href="https://idealists.su/tag/jemalloc" rel="nofollow noopener noreferrer" target="_blank">#jemalloc</a> от <a class="hashtag" href="https://idealists.su/tag/facebook" rel="nofollow noopener noreferrer" target="_blank">#Facebook</a>), но и приколы вокруг узлов <a class="hashtag" href="https://idealists.su/tag/numa" rel="nofollow noopener noreferrer" target="_blank">#NUMA</a> и даже huge pages (снижающих «давление» на <a class="hashtag" href="https://idealists.su/tag/tlb" rel="nofollow noopener noreferrer" target="_blank">#TLB</a>, меньше промахов).Первая же проблема выплыла почти сразу — не реально использовать библиотеку AMQP-CPP с уже предоставляющейся поддержкой <a class="hashtag" href="https://idealists.su/tag/libev" rel="nofollow noopener noreferrer" target="_blank">#libev</a>, <a class="hashtag" href="https://idealists.su/tag/libuv" rel="nofollow noopener noreferrer" target="_blank">#libuv</a>, <a class="hashtag" href="https://idealists.su/tag/libevent" rel="nofollow noopener noreferrer" target="_blank">#libevent</a>. Несовместимы эти очереди сообщений с имеющейся моделью управления потоками и организации задач на агентах.Почему был взят epollПодход используемый в <a class="hashtag" href="https://idealists.su/tag/epoll" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> выглядит более современно, меньше копирований памяти между user space и kernel space. А при появлении данных в отслеживаемом файловом дескрипторе можно напрямую перейти по указателю на объект класса или структуру данных. Тем самым обходиться без поиска дескриптора по индексным массивам/контейнерам. Сразу же работать с экземплярами объектов оборачивающих нужное <a class="hashtag" href="https://idealists.su/tag/tcp" rel="nofollow noopener noreferrer" target="_blank">#tcp</a> -соединение, того самого, в которое и пришли данные.И тут обозначилась вторая проблема, что используема AMQP-библиотека не вычитывает данные целиком из потока сокета. Например, забирает данные лишь до тех пор, пока не насытится автомат состояний (finite-state machine), выполняющий парсинг сущностей AMQP-протокола.Используя <a class="hashtag" href="https://idealists.su/tag/epoll" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> приходится выбирать на какой вариант обработки событий ориентироваться:<ul><li>срабатывание оповещений «по уровню» (level-triggered),</li><li>выбрасывания событий «по фронту» (edge-triggered).</li></ul>И беда с библиотекой в очередной раз показала, что нельзя использовать работу «по фронту» (edge-triggered) не изучив досконально работу подсистемы отвечающей за вычитывание данных из файловых дескрипторов. И появление флага EPOLLET в коде является маркером, о том, чтобы проводить аудит использовавшихся решений.Про Edge Triggered Vs Level Triggered interrupts можно почитать в <a href="https://venkateshabbarapu.blogspot.com/2013/03/edge-triggered-vs-level-triggered.html" rel="nofollow noopener noreferrer" target="_blank">https://venkateshabbarapu.blogspot.com/2013/03/edge-triggered-vs-level-triggered.html</a>)<a class="hashtag" href="https://idealists.su/tag/programming" rel="nofollow noopener noreferrer" target="_blank">#programming</a> <a class="hashtag" href="https://idealists.su/tag/linux" rel="nofollow noopener noreferrer" target="_blank">#linux</a> <a class="hashtag" href="https://idealists.su/tag/трудовыебудни" rel="nofollow noopener noreferrer" target="_blank">#трудовыебудни</a>

Felix Palmen :freebsd: :c64:<a href="https://mastodon.social/@toomanysecrets" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@toomanysecrets</a> It's kind of funny the talk is all about <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> although the text then correctly tells that BSD's (originally <a href="https://mastodon.bsd.cafe/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a> but adopted by the others as well) <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a> was the first async event interface without the scalability issues of <a href="https://mastodon.bsd.cafe/tags/select" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#select</a> and <a href="https://mastodon.bsd.cafe/tags/poll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#poll</a>. 🙈Talking of these, I'd say the "Before epoll" paragraph is, at least partially, wrong. It just omits select/poll. Both are specified in <a href="https://mastodon.bsd.cafe/tags/POSIX" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#POSIX</a> (and therefore available on *many* systems), and both offer the same async model, with the drawback of having scalability limitations. As a rule of thumb, they're fine for a few hundred concurrent clients, but not more.The really wasteful "forking model" (which is kind of classic Unix, really forking one process per client) predates both.It's really a shame there is no common standard for the modern, really scalable APIs. 😞 We have <a href="https://mastodon.bsd.cafe/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a>, <a href="https://mastodon.bsd.cafe/tags/kqueue" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kqueue</a>, there's "IO completion ports" in Windows, etc. I tend to still use classic <a href="https://mastodon.bsd.cafe/tags/select" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#select</a> a lot, if more than a few hundred concurrent (!) clients is nothing you could ever expect for your software, it's still fine.

Martin Bishopepoll: The API that powers the modern internet<a href="https://darkcoding.net/software/epoll-the-api-that-powers-the-modern-internet/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://darkcoding.net/software/epoll-the-api-that-powers-the-modern-internet/</a><a href="https://mastodon.social/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a> <a href="https://mastodon.social/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a> <a href="https://mastodon.social/tags/internet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#internet</a> <a href="https://mastodon.social/tags/FreeBSD" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FreeBSD</a>

GripNews🌘 n8s.site | 非同步 Rust 不壞，問題出在你 ➤ Rust 的非同步問題和生態系統的反思 ✤ <a href="https://n8s.site/async-rust-isnt-bad-you-are" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://n8s.site/async-rust-isnt-bad-you-are</a> 本文討論了使用 Rust 和將非同步關鍵字引入程式碼庫的缺點，提到了當前的問題點，並批評了現在的程式庫和生態系統。 + 文章提供了獨特的觀點，有助於警惕過度依賴非同步編程的風險。 + 這篇文章讓人重新思考了非同步程式設計和相關生態系統的問題，值得深入探討。 <a href="https://mastodon.social/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a> <a href="https://mastodon.social/tags/%E9%9D%9E%E5%90%8C%E6%AD%A5" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#非同步</a> <a href="https://mastodon.social/tags/epoll" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#epoll</a>

Recent searches

Search options

Administered by:

Server stats:

#epoll