Archive for February, 2012
Project Lumberjack to improve Linux logging
In a lively discussion at the RedHat offices two weeks ago in Brno, a number of well respected individuals were discussing how logging in general, and Linux logging in particular could be improved. As you may have guessed I was invited because of syslog-ng, but representatives of other logging related projects were also in nice numbers: Steve Grubb (auditd), Lennart Poettering (systemd, journald), Rainer Gerhards (rsyslog), William Heinbockel (CEE, Mitre) and a number of nice people from the RedHat team.
We discussed a couple of pain points for logging, logging is usually an afterthought during development, computer based processing, correllation of application logs is nearly impossible. We roughly agreed that the key to improve the situation is to involve the community at large, initiate a momentum and try to get application developers on board and have them create structured logs. We also agreed that this will not happen overnight and we need to take a gradual approach.
To move into that direction, the benefits of good logging needs to be communicated and delivered to both application developers and their users.
We also talked about what kind of building blocks are needed to deliver a solution fast, and concluded that we basically have everything available, and even better they are open source. The key is to tie these components together, document best practices and perhaps provide better integration.
Thus project Lumberjack was born, hosted as a Fedora project at https://fedorahosted.org/lumberjack/.
The building blocks that need some care are:
- some applications already produce logs in structured format, those should be integrated (auditd for instance)
- we need to define a mechanism to submit structured logs to local logging services for further processing (ELAPI and some enhanced syslog)
- we need to make sure that local logging services cope with structured data (already available for a long time now)
- we need to define a mechanism to store messages in a structured form and a way query them
- last, but not least we need to define a naming scheme for event data which CEE can bring to the table
Most of these is already possible by using a combination of tools and proper configuration, however learning how to do this is not a trivial undertaking for those who only want to develop or use applications.
Changing that is the primary aim of Project Lumberjack. If you are interested in logging, make sure to check that out.
syslog-ng Insider – February 2012
Dear syslog-ng users,
This is the 11th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news.
Your feedback and news tips about the next issue is welcome at
documentation(at)balabit.com
FEATURED NEWS
Brno: Fedora, CEE, journal and syslog-ng
Last week the Brno Red Hat office hosted two conferences, one small about logging and the Fedora Developer conference. The logging miniconf covered some very hot topics: CEE, journal, auditd and some lesser known projects, like ELAPI. After the formal program, we had some very good discussions about the future of logging.
You can check the diagram drawn up as conclusion here:
http://czanik.blogs.balabit.com/2012/02/brno-fedora-cee-journal-and-syslog-ng/
And read more about how syslog-ng supports CEE: http://algernon.blogs.balabit.com/2012/02/cee-handling-with-syslog-ng/
BalaBit has just released the latest version of its leading log management tool, syslog-ng 4 F2
Adding to the existing, rich feature set which includes high-performance multi-thread processing, encrypted and timestamped log files, disk-based buffering, direct database access, native TLS support, the syslog-ng 4 F2 now supports Application-level Acknowledgement via Reliable Log Transport Protocol (RLTP)™ , a new transport protocol that prevents message loss during connection breaks. In addition, the latest version of syslog-ng can now natively collect and process log messages from SQL databases enabling users to easily manage log messages from a wide variety of enterprise software and custom applications.
syslog-ng 3.3.4 is released
It is a bugfix release, which fixes all previously known problems in the 3.3 series. There is only one change in 3.3 sources since the last release: manual pages were put under the GPL, and XML sources are now also available, so that the entire source code of syslog-ng is free from this point onwards.
Sources are available at http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/download.
Packages for some distributions are available from http://www.balabit.com/network-security/syslog-ng/opensource-logging-system/downloads/3rd_party
Detailed changelog is available at http://www.balabit.com/files/syslog-ng/open-source-edition/3.3.4/changelog-en.txt
Documentation was also updated: http://www.balabit.com/support/documentation/documents/syslog-ng-ose-3.3-guides/syslog-ng-ose-v3.3-guide-admin-en.html/bk01-toc.html
EU Data Protection Directive – How a single regulation could boost the transparency in IT security?
Personal opinion from Balázs Scheidler, CEO of BalaBit
Overall, the EU Data Protection Directive can be a milestone in boosting the transparency of IT security at organizations – similarly to the regulatory compliance regulations after the Enron case. If adopted, the new directive could bring about a change in the implementation of IT security policies so that the current focus on audits could shift to the deeper integration of IT security processes into business processes.
As logging and log management are the base of every monitoring method, technologies with high-speed and zero message loss capabilities, like syslog-ng, will come to the front. Encrypting log files, in which companies store user names, passwords and other sensitive company data is also key to prevent data loss. http://bscheidler.blogs.balabit.com/2012/02/eu-data-protection-directive-how-a-single-regulation-could-boost-the-transparency-in-it-security/
OTHER SHORT NEWS
- syslog-ng web GUI blog updated
- What makes the upcoming v3.4 even more flexible? It’s summarized by Bazsi, lead developer of syslog-ng. Watch out for an upcoming Alpha version!
- There are still many syslog-ng v2.0 users. For those, here is a teaser, what changed since that release
- A syslog-ng presentation for FOSDEM
- ELSA, the high performance web GUI for syslog-ng, received a very useful new feature, post processing of search results
NEW RELEASES
- syslog-ng OSE 3.3.4
- syslog-ng PE 4F2
ARCHIVE
Brno: Fedora, CEE, journal and syslog-ng
Last week the Brno Red Hat office hosted two conferences, one small about logging and the Fedora Developer conference. While our focus was on logging, w also attended part of the Fedora conference and listened to some great presentations.
Cinnamon, Linux tablet, LinuxDevices
For a long time I was a KDE user, but around 4.5 it became too fancy for me and was in my way instead of helping my work. So I switched to Gnome 2. It provided me with a sometimes over simplified, but convenient and consistent GUI. Then suddenly Gnome 3 destroyed the whole thing with a completely redesigned interface. Also, pulse audio became mandatory with Gnome, which has a noticeable impact on sound quality (I have above the average good ears and headphones
)
Neue EU-Datenschutzrichtlinie: Ein Meilenstein für die Transparenz in der IT-Sicherheit
Kommentar von Balázs Scheidler, Geschäftsführer von BalaBit IT Security zu der neuen EU-Datenschutzrichtlinie.
Ende Januar 2012 hat die Europäische Kommission eine umfangreiche Reform der EU-Datenschutzrichtlinie vorgeschlagen, die das Recht auf Schutz der Privatsphäre im Internet verstärken und der digitalen Wirtschaft weiteren Schub verleihen soll. Einige Details dieser neuen Regularien werden sich gravierend auf die künftigen IT- und Sicherheitskonzepte der Unternehmen auswirken und fordern mehr Transparenz der Aktivitäten im Firmennetz.
Die neue EU-Datenschutzrichtlinie könnte sich als Meilenstein auf dem Weg zur Transparenz in der IT-Sicherheit erweisen – ähnlich den gesetzlichen Auflagen für Compliance nach dem Fall Enron. Treten die neuen Direktiven in Kraft, werden sie auch die Umsetzung von IT-Sicherheitsrichtlinien im Unternehmen massiv verändern.
Gegenwärtig investieren Unternehmen hohe Budgets, um gesetzlich vorgeschriebene Compliance-Audits zu absolvieren. Die Compliance-Abteilung wurde hierzu vom IT-Sicherheitsteam strikt getrennt. Dabei wurde jedoch vernachlässigt, dass auch die IT-Sicherheit und der IT-Betrieb von diesem Invest profitieren. So kann es nun passieren, dass ein Unternehmen zwar ein Audit erfolgreich absolviert, im täglichen Betrieb aber persönliche Daten verloren gehen. Denn ein Audit prüft in der Regel nur, ob bestimmte Prozesse vorhanden sind und ob die Beschreibung eines eingeführten Prozesses plausibel und sicher ist. Die praktische Überprüfung bleibt aus Kapazitätsgründen meist aus.
Sicherheitsprozesse in Geschäftsprozesse integrieren
Die neue Richtlinie kann nun endlich dazu führen, dass IT-Sicherheitsprozesse tiefer in die Geschäftsprozesse integriert werden. Denn sie fordern Transparenz, eine Offenlegung und den Nachweis, ob, wann und wie Sicherheitsverstöße stattfinden. Und es folgen Konsequenzen bei Nichtbeachtung.
Die Beantwortung der Frage in Echtzeit “Wer hat im IT-System was getan” wird immer wichtiger. Auch wenn bisher niemand genau weiß, wie die künftige Definition von „schwerwiegende Sicherheitsverstöße“ lauten wird, auf die sich die neue Richtlinie bezieht. Unternehmen sollten sich darauf einrichten, dass es sich sowohl um die Art, wie auch um die Anzahl verloren gegangener privater Datensätze handeln könnte.
Die Forderung, Verstöße „so schnell wie möglich” zu melden, bedeutet im Grunde: „Sobald man von einem Sicherheitsverstoß erfährt“. Die Frage ist jedoch, wann und wer im Unternehmen zuerst informiert wird – der Systemadministrator, der IT-Leiter, der Datenschutzbeauftragte, die Geschäftsleitung oder vielleicht der Dienstleister, der die IT im Outsourcing betreut? Idealerweise erfahren alle zur selben Zeit davon – nämlich über Monitoring-Tools.
Interne und externe Vorfälle in Echtzeit erfassen
Die Erfassung von internen und externen Vorfällen in Echtzeit ist generell wichtig. Ein Schlüsselbereich ist jedoch das Monitoring der Aktivitäten von Anwendern wie IT-Administratoren, die privilegierten Zugang zu sensiblen, geschäftskritischen Daten haben. Um deren Zugriffe zu überwachen und sicher zu stellen, dass die aufgezeichneten Aktivitäten im Nachhinein nicht verändert oder gelöscht werden können, sind Monitoring-Tools unabdingbar. Wie etwa die Shell Control Box von BalaBit. Sie unterstützen dabei, die Ursache von Sicherheitsvorfällen zu ermitteln und Unternehmen in die Lage zu versetzen, aus früheren Fehlern zu lernen. Das will auch die neue EU-Richtlinie erreichen.
Die Meldung über Vorfälle soll „falls machbar innerhalb von 24 Stunden” erfolgen, fordert die neue Richtlinie. Laut der jüngsten Studie von Verizon, dem Data Breach Investigation Report 2011, bleiben in Unternehmen Sicherheitslücken oft Wochen oder sogar Jahre lang unbemerkt offen. Die Ermittlung kleiner Vorfälle und Stichproben hat keine Priorität. Durch die neue EU-Richtlinie sind Unternehmen gezwungen, diese Haltung zu ändern – zumindest im Hinblick auf den Verlust von Personendaten. Logs und Log-Management sind die Grundlage jeder Monitoring-Methode. An Bedeutung gewinnen jetzt Log-Systeme, die große Datenmassen schnell und ohne Verluste bewältigen und lückenlose Ergebnisse liefern können.
Verschlüsselte Log-Dateien, in denen Firmen die Anwendernamen, Passwörter und andere sensible Daten speichern, sind ein Schlüsselelement, wenn es darum geht, den Verlust von Daten zu vermeiden, da diese Dateien nicht einfach zu dechiffrieren sind.
Mit den Lösungen von BalaBit IT Security können Unternehmen den Anforderungen der neuen EU-Datenschutzrichtlinie gerecht werden: Mit der HSRL-Version (High-Speed Reliable Logging) des Log-Systems syslog-ng, die bis zu 650.000 Log-Nachrichten pro Sekunde verarbeiten kann und seiner Shell Control Box.
Über BalaBit IT Security
Das Unternehmen wurde im Jahr 2000 in Budapest (Ungarn) gegründet und beschäftigt Stand 2011 rund 120 Mitarbeiter. BalaBit ist mit einem breiten Partnernetzwerk weltweit tätig und unterhält Niederlassungen in Deutschland, USA, Frankreich, Italien und Russland. Das Headquarter sowie das Entwicklungs- und Support-Center befinden sich in Ungarn. BalaBit ist auf die Entwicklung Proxy-basierter Gateway-Technologien spezialisiert, es bietet Lösungen für die Kontrolle und Auditierung privilegierter IT-Zugriffe und das Log-Lifecycle Management. Die Produkte sind bei führenden Unternehmen aus den Bereichen Finanzdienstleistungen, Telekommunikation, Luft- und Raumfahrt sowie dem Gesundheitswesen im Einsatz. Zu den Kunden zählen zudem Behörden und öffentliche Einrichtungen. BalaBit vertreibt seine Produkte über ein weltweites Partnernetzwerk.
Im Markt bekannt ist BalaBit zudem als die “syslog-ng-Firma”: Die Log-Server-Anwendung, die als Open-Source-Software zur Verfügung steht, ist weltweit bei mehr als 650.000 Kunden im Einsatz und hat sich zum De-facto-Industriestandard in diesem Bereich entwickelt.
www.balabit.com
Pressekontakt
punktgenau PR
Christiane Schlayer
Fon +49 (0)911 9644332
christiane.schlayer@punktgenau-pr.de
www.punktgenau-pr.de
Kontakt zu BalaBit IT Security
BalaBit IT Security GmbH
Dietmar Wilde
Stefan-George-Ring 29
81929 München
+49 (0)89 9308 6477
dietmar.wilde@balabit.com
www.balabit.com
What Zorp is good for
A marketing specialist would claim that it is “good for everything”. Not being one of them, we would rather say that Zorp is not the philosopher’s stone, however, it can solve almost any issue that can be expected from a deep protocol analyzer proxy firewall. The most important cases are the following:
Access control
Access control is a basic functionality of proxy firewalls, but Zorp has an extra feature compared with other firewall suites. Access to the services can be controlled by the attributes of lower layers of the ISO/OSI model, like IP addresses or ports, but in case of Zorp there is a possibility to define sets of IP subnetworks, called zones. Zones are IP subnetwork groups that administratively belong together (for example all those who are permitted to access FTP servers for upload) and can be linked to a tree hierarchy. Access control rights are inherited between the levels of the zone tree. A top-level access (for example a right to download from FTP servers) is in effect in the lower levels as long as it is not blocked. In this way an administrative hierarchy can be created that is independent from the network topology and the location of the devices, while reflecting only the network policy.
When an access control policy is being created, we first have to find answers to the “who”, “what” and “how” – questions. Resources should be accessible only for a specific group of users under the defined conditions. It may mean that each request and response must be recorded to the system log when a given server is accessed. Some features of the protocol (for example: STARTTLS in case of SMTP) causing incompatibility between the client and the server may have to be filtered out. Some items of the protocol (for example PUT in case of FTP) may be rejected. Some protocol items (for example user-agent in case of HTTP) may be changed to avoid information leak. Secure connection may be decrypted on one side and encrypted again on the other side. The following sections will describe this in detail.
Information Leak Prevention
Several protocols leak information about the running softwares, the networking options of the clients, which is usually not filtered or not blocked by the firewalls, because they are absolutely compliant with the related standards. An example of this is the user-agent header in the HTTP protocol, which contains the name and the version of the web browser connected to the server. In this case an information about the software being run on the client machine is received by the visited web server without the knowledge or the permission of the user.
The proxy settings of the web browser, the IP address of the machine, the URL of the previously visited web page (referrer of the currently visited one) are leaked in the same way. Similar methods exist in case of several protocols, besides HTTP. System administrators have to be aware of these type of information leaks and have the means to forbid them. Zorp is an easy-to-use and flexible tool for that.
Interoperability
Continuing the example above, not only forbidding of complete protocol items is possible, but also the modification of their values. It can solve the problem of the interoperability for example when a web server constraints the type or the version of the connecting browser despite of the fact that it has no good or valuable reason. Such a situation can be solved easily by changing the value of the user-agent header in the request sent by the browser to a value which is acceptable to the server.
The lack of encryption support may cause interoperability mainly in case of old-fashioned software especially when the traffic should pass through an untrusted network. There are several solutions to this problem, but if we want to proxy the traffic and use different methods of encryption (STARTTLS, SSL) to the client and the server, Zorp is still one of the best solutions. It is possible to establish an encrypted connection through the untrusted network and a plain connection through the trusted one. It is also possible the use different versions of encryption (TLS 1.0, TLS 2.0) to the client and server.
To do that, capability of establishing encrypted connections separately to the client and the server is necessary, but not sufficient. The reason is the way to upgrade a plain text connection to an encrypted (TLS or SSL) one instead of using a separate port for encrypted communication (STARTTLS), where understanding the protocol is a must. If we want to hide this functionality from the client and the server even if both of them support it, to solve an incompatibility problem, Zorp can help us. We can conceal features of the clients or the servers (for example STARTTLS in SMTP, or compression in HTTP) from each other.
To continue the encrypting example, Zorp can hide the STARTTLS feature of the SMTP server from the client, which prevents to initiate encrypted communication in this way. Certain combinations of client and server side SSL settings (for example when SSL is forced in server side) Zorp does it automatically.
Content Filtering
Content filtering is a key feature of firewalls. Zorp is not an exception to this rule, even if without extensions there are only limited opportunities to do that work. However, each of spam filtering, virus scanning, URL filtering is possible by means of external software components. Let the cobbler stick to his last. Zorp does nothing else, but analyzes the protocol to find the particularly interesting parts of the traffic (URL, downloaded data, e-mail attachment, …) and passes it to the necessary application. As the result of the content filtering and possibly other conditions, Zorp may accept, reject or only log the request, or even quarantine the response. We have nothing to do, but establish connection between the Zorp and the chosen content filtering software (for example: ClamAV, SpamAssassin, …) with a simple adapter application, which makes the location of the data known to the content filtering tool and forwards the result to Zorp.
Audit
Establishing an access control system is only the first step on the way to achieve a well-controlled and secure network. Operating and administrating this network is more difficult. Above all, we need to know what is happening in our network, because only this information can create the possibility to improve the access control system. On the one hand we have to answer what kind of events have violated the current network policy. On the other hand we are in need of the information whether a permitted action has happened or not and if so, than how. Zorp is able to log the necessary information in both cases.
The benefit of Zorp is the fact that we can retrieve information from the proxies in application level so events of the network can be handled in the application level also. Even requests and responses of a protocol can be recorded to the system log, which can be very useful in case of an audit. After the necessary configuration of the proxy from the log messages it can be proved whether an event has happened or not in a specific time interval and also statistics can be created based on them.
Flexibility
Zorp is able to solve the general uses mentioned above as it is, but the strength of the Zorp lies in the fact that it is easily extendable and customizable to solve specific problems. We do not need to reimplement any kind of functionality, especially the protocol analyzers, we can reuse and extend them to meet our requirements. Nevertheless the proxies are mainly written in C, they can also be scripted in Python with all of the benefits of the language. Existing ones (HTTP, FTP, …) can be specialized, or a new one can be implemented if we want to analyze the protocol at application level only. It is possible with a special kind of proxy (AnyPy) which does anything, but the application level analysis, so we can focus on that job.
Official Zorp GPL support page can be found here.
Ready-to-use virtual machines sporting Zorp GPL can be downloaded here.
The content of blog post can be used freely under the terms of Creative Commons Attribution-ShareAlike 2.5 license.
Journal and syslog-ng – revisited on Fedora
Last week I gave journal a try on openSUSE. I ran into a bunch of problems, both systemd/journal and packaging related. So I decided to give it a try also on Fedora Rawhide. Summary: problematic, but in another way
My post mentioned
Cool howto, which uses my apache logging solution:
https://github.com/ClockworkNet/apachelogging
Since I wrote my post I have changed it a bit: I use a file for logging and I rotate it. It is a “native” disk-buffer, however it is not as professional as the on which is in syslog-ng PE.
Training dates 2012Q1
It’s been a long time I haven’t posted. I will write more posts soon. First let’s see the training dates for 2012. First items just for the record;-)
- 26-26 Jan: SCB-HU (location: Budapest)
- 31 Jan- 1 Feb: SCB-EN (location: Paris)
- 8 Feb: SR-HU (location: Budapest)
- 9-10 Feb: BCZA-HU (location: Budapest)
- 14-15 Feb: SCB-EN (location: Istanbul)
- 22-23 Feb: SPE-HU (location: Budapest)
- 24 Feb: SSB-HU (location: Budapest)
- 7-9 Feb: BCZA1-HU (location: Budapest)
- 13-14 Feb: SCB-HU (location: Budapest)
- 22-23 Feb: BCZE2-HU (location: Budapest)
So you can record we become an international company.
I will have a hard Q1. I promise I will post soon.
Twitter
LinkedIn