[Linux-aus] contest proposal

[jon.maddog.hall@gmail.com]

Russell,

I was once a beekeeper.  No single bee makes an entire cell in the honeycomb.  Each bee deposits a bit of wax, pinches it with its mandibles, and moves on.  Eventually an entire cell is finished, a cell that makes the honeycomb one of the most perfect structures for storage capacity, strength and economical use of materials (wax) in nature.

[As before, people not interested in history can stop reading here.]

When the DEC Alpha was being created it was one of the first commercial truly 64-bit systems to run Unix.  DEC concentrated on the scientific/engineering market, so an efficient libm(3) meant a lot to our customer base.

DEC hired a mathematician to optimize libm(3) to make it the fastest in the Unix world, and it was.   Consequently DEC did not supply the source code to libm to our customer base, in fear that DEC's competitors would see how the optimizations worked and therefore make a library that was faster than DEC's.

When the Linux/Alpha port was happening the developers wanted DEC to give them the sources to libm(3), but DEC refused.   DEC was willing to build a binary for Linux/Alpha, but not give the Linux community the sources.

I was caught in the middle, between DEC's management and the Linux community.

Finally I emailed the community saying "If you are such hot-shot programmers, why don't you create a BETTER libm?"

Silence from the Internet.

Then a few days later an email message: "sin is 2% faster".  A couple of days later "cos is 1.5% faster".

Subroutine by subroutine the libm routines on Alpha were reprogrammed to be faster than DEC's.

It was then I began to realize the power of the community.

[End of DEC history]

The point to all of this is that I would not expect a wave of a wand and all of the code will be smaller and faster.

/* So this change to "memory's so cheap I can treat it as free" ends up impacting a lot more than just the memory, speed, and power consumption.*/

I agree, the impact may be a lot of things, and affect a lot of systems.

However each small improvement (like that of the honeybee) might build into a much larger improvement (the honeycomb), or at least slow down the growth of inefficient code.

Just focusing on the problem might help.  Without focus people will not pay attention to the issue.

There are other examples than libm.   Remember when Linus forced the re-write of the I/O system to make it much more structured, and cleaned up all the "cruft" in the device drivers?   We lost a lot of redundant code in the device drivers.

Commercial companies do not often do this, because they are focusing their engineers on new features to satisfy customers.   FOSS can do it all....new features that are efficient and make the old code more efficient.

I think Russell Coker's suggestion has merit.   I think the hardest part will be to figure out the "reward" system, even if the rewards are small.   I could also see having people who consistently contributed to this effort be promoted to "hacker" (remember the origin of the word "hacker") and perhaps earn a sponsored speaker's position at the Australian Linux Conference.

Who are the judges and how are the rewards given?

I could envision a whole undergraduate curriculum class in computer science on "optimizations", with term projects given to optimize FOSS code.

Warmest regards,

maddog
> On 01/01/2024 7:23 PM EST Russell Stuart via linux-aus <linux-aus at lists.linux.org.au> wrote:
> 
>  
> On 2/1/24 09:18, jon.maddog.hall--- via linux-aus wrote:
> > The DEC OSF/1 Alpha system reduced from 64 MBytes to 32 MBytes 
> > actually benchmarked as 7% FASTER than before all this work was 
> > done.
> 
> This rings true, but nonetheless attempts to save on code size are
> swimming against the tide today.  We seem to be in the process of
> abandoning sharing code in libraries via DLL's.
> 
> I think this was popularised by Google, when they statically linked
> their binaries.  It makes a lot of sense in an environment when you give
> each task it's own isolated VPS, each VPS sharing the same kernel.  Its
> easier (and in my experience more robust) than sharing a docker container.
> 
> It's now moving into our languages.  The increased CPU speed has allowed
> the rise of interpreted languages.  They are not only slow CPU wise,
> multiple web pages in a browser generally means multiple copies of the
> shared javascript libraries.  Newer compiled languages like Go, Swift
> and Rust all favour statically compiled binaries.  Rust in particular is
> wedded to it because of it's love of monomorphization.
> 
> Gradually these new languages are worming their way onto out Desktop
> machines.  But unlike Google's VPS's, I'm pretty sure statically linked
> binaries are a net negative there.  It's not only bloat (think: every
> program contains it's own copy of libc).  It's when a bug is found in
> libc, the distro has to recompile and redistribute every binary that
> uses it.  Security updates become more like what you see on phones:
> gigabytes, every time.
> 
> Maybe that's possible.  But when every program was forced to share one
> libc, it meant there was one version of libc on the system.  When that
> isn't enforced programmers tend to believe it's their right to depend on
> a particular version of every library they use.  We see the attitude in
> most prominently in Javascript, but Rust's beautifully crafted
> versioning system encourages the same thing.  But from a distro's point
> of view this is a disaster.  Instead of tracking security flaws in one
> version of a library, they now have to track them in every version of
> the library from the year dot.  I doubt that is possible.
> 
> So this change to "memory's so cheap I can treat it as free" ends up
> impacting a lot more than just the memory, speed, and power consumption.
> _______________________________________________
> linux-aus mailing list
> linux-aus at lists.linux.org.au
> http://lists.linux.org.au/mailman/listinfo/linux-aus
> 
> To unsubscribe from this list, send a blank email to
> linux-aus-unsubscribe at lists.linux.org.au