Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
×
Open Source Security

Will It Take More Than Open Source Funding To Prevent the Next Log4j? (openssf.org) 110

"While the lack of funding in open source is certainly a problem, could funding have prevented the Log4j vulnerabilities?" asks Mike Melanson's "This Week in Programming" column. "Would funding actually prevent similar vulnerabilities in the future...?"

Or is that an oversimplification? In a blog post for the Linux Foundation's Open Source Security Foundation (OpenSSF), Brian Behlendorf argued that open source foundations must work together to prevent the next Log4Shell scramble, outlining seven points that OSS foundations could do to mitigate security risks. Among those seven points — which include security scanning, outside audits, dependency tracking, test frameworks, organization-wide security teams, and requiring projects to remove old, vulnerable code — not once was funding mentioned. Rather, Behlendorf precedes these points by saying that "Too many organizations have failed to apply raised funds or set process standards to improve their security practices, and have unwisely tilted in favor of quantity over quality of code."

Behlendorf continues after his list of seven suggested acts with a section that boils everything down perfectly:

"None of the above practices is about paying developers more, or channeling funds directly from users of software to developers. Don't get me wrong, open source developers and the people who support them should be paid more and appreciated more in general. However, it would be an insult to most maintainers to suggest that if you'd just slipped more money into their pockets they would have written more secure code. At the same time, it's fair to say a tragedy-of-the-commons hits when every downstream user assumes that these practices are in place, being done and paid for by someone else."

Behlendorf does go on to make some points about funds and fundraising, but his point is less on the lack of funding than the allocation of those funds and how they need to be focused on things like paid audits and "providing resources to move critical projects or segments of code to memory-safe languages, or fund bounties for more tests."

Behlendorf says that, in the new year, the OpenSSF will be working to "raise the floor" for security in open source.

"The only way we do this effectively is to develop tools, guidance, and standards that make adoption by the open source community encouraged and practical rather than burdensome or bureaucratic," he wrote. "We will be working with and making grants to other open source projects and foundations to help them improve their security game."

Behlendorf was a founding member of the Apache Group, which later became the Apache Software Foundation.

So as a long-time member of the Open Source community, he calls the Log4j vulnerabilities "a humbling reminder of just how far we still have to go."
This discussion has been archived. No new comments can be posted.

Will It Take More Than Open Source Funding To Prevent the Next Log4j?

Comments Filter:
  • the next log4j (Score:4, Interesting)

    by peterww ( 6558522 ) on Saturday December 25, 2021 @12:48PM (#62114343)

    Will be in some Go library, and all those statically compiled Go apps will have to be rebuilt and reinstalled one at a time. Will be much worse because there's no automation for remediating it and it will affect more applications

    • by sodul ( 833177 )

      Are you trying to complain about Golang here?

      The log4j issue did require the java services using them to be rebuilt, the war and jar files usually embed the faulty libraries and require the equivalent to a full rebuild and re-install. The static nature of Go does not make the process much worse than with Java TBH.

      With java: update your log4j version in maven/ant/graddle, deal with your transient dependencies, commit, rebuild and redeploy your binaries.

      With Golang: update your log4g (or whatever equivalent)

      • Dynamic versus static linking is IMO not a huge issue for FOSS. You have the source. You can update dependency lists, recompile, test, and be done, at least for now.

        But with proprietary software, especially if it's statically linked, you're kind of at the mercy of the vendor. It is the only entity capable of updating the libraries and then rebuilding. If it doesn't, for whatever reason, you're libFoobar.so'd.

    • The reality is that despite shared libraries saving us from security exploits, so often they're used in monolithic components where they are essentially not shared libraries because each component has its own local shared library. So you're still patching stuff one at a time, not just replacing that one library.

      I'm not at all convinced that shared libraries are really any worse than static compilation. While patching a statically compiled application may be a headache, at least you'll know it won't bomb o

      • If it's the same version of the library as before with a 2-line patch, and the devs aren't terrible / the codebase isn't awful, it _should_ be fine to patch shared libraries in place. But yeah, sometimes it's not.

        Where shared libraries are much better in this regard is in how standardized they are. With shared libraries you can 'apt-get update && apt-get upgrade' (or however the library is packaged) on your development machine, test it locally, then promote the change to production.

        With statically c

        • I guess I get it in an environment where you or your org is doing the development and deployment across a common platform where you actually use/control the libraries.

          But as a regular joe IT administrator, I just see 23 different platforms from a bunch of different vendors where there is no concept of just replacing a library. I mean there is in some very narrow cases IF you want to fuck around with a vendor's supplied appliance/VM/platform where you're not expected to replace libraries and its not support

      • I agree with you (I think). The real hassle is getting the individual coder to wake up, switch context, go dig up the impacted version from their VCS, get it running in a dev environment the same as the impacted versions, fix/patch/workaround it, then re-release the patched/fixed/worked-around version, then re-package it (hopefully with little or no change but gotta test/check), lastly push it downstream to repos etc... Very little of that process and the steps I'm leaving out for brevity is actually coding
        • The people who crow about their advantages so much seem to be very dev-focused and involved in fairly integrated environments where they *can* just replace a library and fix "everything" that uses the affected library.

          The rest of us (OK, me) live in this fragmented universe of hardware devices and software where the benefits are pretty elusive because the patch process isn't just injecting a new library, even if the vendor patching process does this, and it tends to be just as "involved" a process for outri

          • Oh oh oh, and I almost forgot to comment on your point about containerization and microservices reproducing all the damn shared libraries again anyway, totally washing away most of the shared library advantages. This is a very under-rated point that I often bring up also. What's old is new again. Why bother when compiling a Docker service or some such? I don't see the point of using shared libraries at all in that situation. Just seems like 99% downside at that point.
            • Yeah, it's mostly of real world value to developers and systems which are running in a common environment that actually use a shared library.

              But most of real world IT doesn't work that way. Hardware appliances (many running full-on operating systems anymore), virtual appliances, virtualization generally, containers (although I'm not sure if some container schemes don't just use the host system libraries, so maybe some value there) all seem to make shared libraries kind of a joke, a savings of a few gigs of

              • Again, I agree. To whit, our embedded systems are pretty much built as a monolith anyway. Sure we have packages, but the build for our software touches enough of the system (and that system is small enough) that we just rebuild the whole thing and re-run our unit testing scripts looking for any deviations from the last run mostly with simple mechanisms like diff. I don't pretend everyone could do that, but on the other hand I suspect what we do isn't much different from what other embedded developers do. We
  • by Arethan ( 223197 ) on Saturday December 25, 2021 @12:53PM (#62114359) Journal

    It's not just a money issue, it's a time and direction of effort issue. Lots of open source isn't build for purpose on a schedule, but instead is built for fun or convenience taking as much time as it needs or wants.

    Behlendorf does go on to make some points about funds and fundraising, but his point is less on the lack of funding than the allocation of those funds and how they need to be focused on things like paid audits and "providing resources to move critical projects or segments of code to memory-safe languages, or fund bounties for more tests."

    Java is already "memory safe" - that was not the problem. Uncovering inventive ways to abuse design flaws is going to be devastating regardless of who finds it first - once details of the discovery are leaked, it's a race between patches and exploit kits - so lack of paid auditors was also not the problem. Jamming functionality into JNDI and log4j with more regard for capability than security was the problem - the only way we fix that is by encouraging sufficient security paranoia during the design phase of the tooling and the tool selection phase of the downstream consumer. I don't quite see an obvious place to inject money to fix that.

    • And commercial funding by careless funders can lead to extraordinary destabilization as those funders seek their own, distinct goals.

    • When it says "batteries included", it just means that someone can use those included batteries to start a fire.

    • It's not just a money issue, it's a time and direction of effort issue.

      It hasn’t even been established that the log4j issue had anything to do with money whatsoever. Well-compensated programmers can have bugs in their software, same as anyone else.

      As I posted the last time this same topic came up: The active developers appear to be employed professionally as coders. Are they doing log4j as part of their regular jobs? Or is this a side project, done on their own in their own free time? If the former, then using log4j as part of this “open source needs more funding

    • by mbkennel ( 97636 )

      > Jamming functionality into JNDI and log4j with more regard for capability than security was the problem - the only way we fix that is by encouraging sufficient security paranoia during the design phase of the tooling and the tool selection phase of the downstream consumer. I don't quite see an obvious place to inject money to fix that.

      Paid auditors, with the right capabilities.

      People, partially outside the normal development stream, whose skill and experience centers around knowledge about how seemingl

      • Actually we already have that. Pretty much all of the largest tech companies (apple being the notable exception) have teams of people on their payroll whose only job is to find security vulnerabilities in open source software (in google's case, even closed source,) or alternatively, they're major financial backers of organizations that do, like ISRG.

        • And why would Apple be an "notable exception", when Apple is the spearhead of OpenSource Software funding and improvement?

    • by gweihir ( 88907 )

      Jamming functionality into JNDI and log4j with more regard for capability than security was the problem

      Exactly. Functionality where using it can so easily go wrong has no place in a language that need to support secure coding. No dangerous functionality should ever be this easy to use. The log4j people were just the first victims of this glaring design flaw.

  • What highly funded computer system has not had security vulnerabilities?

    If there are ever vulnerabilities in things like Cisco routers, which there have been, what makes you think any amount of money could secure every open source library 100% of the time?

    This is why defense in depth is so important, if an internal system becomes compromised it shouldn't leave you wide open. And also probably why a range of technologies for internal systems is good, so one library vulnerability doens't leave every single w

    • Cisco routers have _deliberate_ vulnerabilities, backdoors deliberately installed for NSA se. Deliberately insalling backdoors means the system will be vulnerable, even if the other code is good.

                    https://www.tomshardware.com/n... [tomshardware.com]

      • by gweihir ( 88907 )

        Cisco routers have _deliberate_ vulnerabilities, backdoors deliberately installed for NSA se. Deliberately insalling backdoors means the system will be vulnerable, even if the other code is good.

        https://www.tomshardware.com/n... [tomshardware.com]

        There is no indication the majority of serious vulnerabilities found in CISCO gear in the last few years was deliberately placed. Too many were too easy to find or use. Not even the NSA is that stupid. If the NSA were to place backdoors (which is possible) they would at least make they are hard to find or use for others. Or they would use "NOBUS" backdoors in the crypto in the first place, such as they had placed rather obviously in Dual_EC_DRBG. Too many of those CISCO vulnerabilities made the gear open t

        • Cisco security is fairly pointless with so many backdoors, and their somewhat secret sharing among crackers. See articles about or read the revelations by Edward Snowden of Cisco embedded vulnerabilitities.

          https://www.infoworld.com/arti... [infoworld.com]

          Cisco claimed not to know about the program. That's very difficult to believe.

          • by gweihir ( 88907 )

            So you mean Cisco stopped caring about security at all with the placed backdoors there? Possibly. But they may also just be incompetent.

    • by gweihir ( 88907 )

      But we are a long way from treating computer security as seriously as we should be, and just throwing money at it will not help at all.

      Indeed. One problem is education. I currently teach an application security course for BA CS students. This course is elective. And you know how many mandatory courses in secure software the students have overall? Absolutely zero. And this in 2021. It is a complete disgrace. And it is the same on other places. And these are not bad CS programs at all, apart from this one glaring flaw they are pretty good. But people that do not even know what makes things vulnerable cannot prevent things from being vulnerab

    • What highly funded computer system has not had security vulnerabilities?

      The issue here is that this isnt a computer system, nor is it an application the end user wants to run.

      Its a logging library.

      Why is a logging library so complex that it cant be trivially secured?

      There is no fucking excuse, and the blame doesnt land so much on log4j, but upon the developers that chose to use such a library.

      • Gross violation of the single-responsibility principle.

        IMO, a logger should just log. Nothing more. No encrypting home directories. No managing system time. No washing dishes, no seeding PRNGs, no editing text. Just log.

        If that were the case it probably could have been made a lot more secure. So would other well-known and even more problematic pieces of software.

        A slight variation on "Zawinski's Law" seems to be in effect here. Every program seems to expand over time until it can exploit vulnerabilit

  • But lets just keep talking about it in the entire article...
  • corporate controlled and driven since the beginning(Sun, Oracle). That was one of the reasons I chose to stay away back in the day. Java = Open Source! you know I never really believed that when it came to Java..
  • It comes down to this: Every job you don't do, you have to make sure that someone else does it to your liking.

    If you use a third party hardware supplier, you get warranties, certificates, specification and other material, that allows you to vet the stability and usability of the product you buy. But if you are a shop that wants a certain quality, you run an incoming inspection on your supply, and if you run into failures, you talk to your supplier or switch to another one.

    You should do the same for a th

    • I think this really more than anything else. I think it is easy to fall into the trap of software is easy to update. Problem is, you forget it is even there in modern large systems. I double checked to make sure that somehow I wasn't using log4j even though I never heard of it.
    • When I use open source software at my job, I look at the code to determine whether it's secure enough, in the rights, for the purpose I intend to use it for. Often a improve some part or make it more secure. That costs my time, which is my employer's money.

      The summary lists the following points:
      security scanning, outside audits, dependency tracking, test frameworks, organization-wide security teams, and requiring projects to remove old, vulnerable code

      You can have security scanning done by paying so

    • Actually, every job you don't do requres that you trust someone else has done as well as you. If you're a terrible programmer, then that bar isn't very high.

  • by gweihir ( 88907 ) on Saturday December 25, 2021 @02:34PM (#62114535)

    The problem is the lookup-mechanism provided by Java. That this was an exceptionally bad idea security-wise was entirely clear. The log4j people merely had the misfortune to get hit first. And there will be more such strokes of genius in Java. It is time to scrap it as a failed approach.

  • by Todd Knarr ( 15451 ) on Saturday December 25, 2021 @02:58PM (#62114603) Homepage

    Log4Shell wasn't caused by any bug. The software was broken as designed. It wasn't even one flaw in the design, it was the interaction between two flaws:

    1. A deliberate decision was made to parse everything being substituted into the format string as another format string. This made it impossible to avoid interpreting raw user input at some point.

    2. A deliberate decision was made to permit format strings to use calls to a naming system that could retrieve data without any limits on where it was coming from. This made it impossible to prevent malicious data from being pulled in.

    Odds on these decisions were made by 2 different people who didn't have any contact with each other. If Behlendorf wants a place to start, his foundation can start with providing that kind of overview analysis of common libraries looking for places where design decisions interact in undesirable ways.

    Note: one major cause of those interactions is the drive to make code as simple and consistent as possible. You can see that in the first decision: all strings are the same, there's only one function for handling them and that function just gets called recursively every time something more needs to be pulled in. The interaction can only be stopped by adding complexity to the code: making the formatting function distinguish between two different types of strings (format strings that need parsed, and data strings which must not be parsed), and modifying the naming system to allow the caller to restrict the domains it can resolve names for to only those that are trusted (which will vary from application to application). I see that all too often from people who don't want to admit that there isn't a clean, simple solution that's safe to use, that the only acceptable solution is going to be messy and awkward and have far too many special cases.

    Oh, also beware "Well, that can't ever happen.". If it truly can't happen, your tools will flag the code as unreachable. If it isn't flagged as unreachable, then it can indeed happen if something goes sufficiently wrong (see https://www.cl.cam.ac.uk/~rja1... [cam.ac.uk] for a definition of "sufficiently wrong").

    • Where are the 1000 eyes looking at this decision and calling it out though ? Even I can see it is a bad idea when explained the way it has been yet it was not caught in years. The libraries are just too complex. A bare bones RFC-compliant logging component would probably suffice for almost all applications, it just does not need all the bells and whistles.
      • Because a lot of times a bare-bones RFC-compliant library won't suffice. Format strings for instance are a good thing for applications, they let the logging library separate the original data that went into the rendered message so logs can be searched by the unrendered message template (catching all occurrences regardless of the values substituted in) or by particular substitution field values. Name lookup is good because often you want to know the value of the name at the time the problem occurred as oppos

      • Where are the 1000 eyes looking at this decision and calling it out though ?

        This was found by the 1000 eyes looking at it.

        • This was found by the 1000 eyes looking at it.
          No, it was not.
          The flaw was in existence for roughly 10 years!! And got found by four eyes "playing Minecraft!"

          • https://en.wikipedia.org/wiki/... [wikipedia.org]

            No, it got found by someone looking at the code. The Minecraft thing came after.

            Also, ten years is irrelevant. No one said everything had to be found immediately. The fact is, it was found because someone could look at the code, and they did. Simple as that.
            • Well,
              it is all over the news that it was discovered by 2 Mindcraft players.
              No idea if the Alibaba cloud guy in Wikipedia is one of those Mindcraft players.

              Fact is: it was not discovered by "1000 eyes" looking on the code: because before it was discovered, no one looked over the code.

              The fact is, it was found because someone could look at the code, and they did. Simple as that.
              Exactly. However parent was of different opinion.

              • And then it was all over the news that China suspended contracts with Alibaba for not reporting it to them first. The Alibaba guy found it first, because that was his job (or his team's job, assuming he worked as part of a team).

                Fact is: it was not discovered by "1000 eyes" looking on the code: because before it was discovered, no one looked over the code.

                That's as ridiculous as saying "why is it always in the last place I look?"

                Pray tell give me an example of someone finding an exploit before looking at the code.

                • Pray tell give me an example of someone finding an exploit before looking at the code.
                  Erm, no idea about what you want to nitpick.

                  Our parent claimed: 1000 eyes found the bug. I pointed out, no: it was 4. Now you reduce it to 2 who found it first and 4 who found it later.

                  So: what is the reason you interjected into this discussion?

    • by tlhIngan ( 30335 )

      Odds on these decisions were made by 2 different people who didn't have any contact with each other.

      Well, format string substitutions is extremely useful for logging - it lets you log a message explaining the problem and then filling in various details, which is hard to do without formats. Some of them are simple - you might want a debug message to tell you the file/class/method/line where it's being printed, but you also might want to log stuff like the user making the request (depending on the level you

  • KISS (Score:4, Informative)

    by bradley13 ( 1118935 ) on Saturday December 25, 2021 @04:35PM (#62114847) Homepage
    Here's an idea: stop addi g every possible feature to things. Everybody's ideas get added, because...why? Lean, simple code instead of megabytes of cruft. That's what OSS ought to emphasize. Personally, I avoid using Apache libraries, precisely because every library is huge and complex, and every library brings in additional libraries, equally huge and complex.
    • Why? The problem is somewhat an evolutionary problem. There have without a doubt been many, many simple logging libraries that have met the needs of a handful of users. In order for a library to be installed on lots of systems, it has to meet the needs of lots of users, which leads to complexity. "Natural selection" favors the complex libraries. It is similar to why an application like Word has so many features. Each user might use 5% of the application but the 5% is different for each user. All the feature

    • At the very least, stop enabling everything by default.
  • Doesn't this vuln require outbound route to a hostile server? Isn't the mitigation default deny outbound which should be default?

    • Doesn't this vuln require outbound route to a hostile server?
      Correct, unless you are an insider and want to hack it as an employee etc.

      Isn't the mitigation default deny outbound which should be default?
      Exactly. And that actually be configured on the host running the VM already, or any router/gateway leading outwards.

  • I don't see it as a funding issue, and I don't believe funding would have prevented log4shell.

    There were obviously resources available to develop the feature that was requested. Also, there were obviously resources available to quickly fix log4j. I seriously doubt that if they had gotten a bag of money five years ago, they would have spent it looking for this kind of vulnerabilities.

    Also, I don't see the vulnerability as such as an OSS issue. Similar things happen in closed source commercial software, small

  • In High-Assurance Design (https://www.amazon.com/High-Assurance-Design-Architecting-Enterprise-Applications/dp/0321793277), I explained by standards and bureaucratic approaches will not work, and why the only thing that works is making security a priority, and requiring all developers to learn secure coding as a core expertise. And I believe that will only happen if software producers bear significant liability for the losses due to vulnerabilities.
  • What app/computer anywhere is a prime example of a secure system? Is there even one in the world that can serve as an example? The general consensus seems to be that if there is a will there is a way to get into any computer that is networked and to compromise any app, and the fact that provable security is still an open scientific research question is an indication that security is not yet possible. Apps in particular are a huge vulnerability everywhere. Until we can build one secure networked app/comp
  • Programmers don't have to ask your permission to write code. Surprise! They can write whatever the fuck they want and the hapless assholes complaining about it can feel free to run that code, or fuck off. That's your choices when you are a taker (and a whiner, I might add) rather than a maker. I guess you could all get together and try to ban independent coders, which I see many have already suggested. Disgusting.

Top Ten Things Overheard At The ANSI C Draft Committee Meetings: (10) Sorry, but that's too useful.

Working...