Monday, August 10, 2009

Major differences in TCP/IP stack from 2.4 to 2.6 kernel

I Got lot of comments on the book saying that the book is not up-to-date. This is partially correct and it is all in the mind. I have myself looked into 2.6 latest networking kernel related to TCP/IP. I found that except for the few features, major algorithms remain the same. If I take route lookup algorithm, I really don't see any. So, performance as far as route lookup is concerned should not vary much as data-structure remains the same for the routing table and Linux still implements hashtable. If I see major difference, it lies in the locking mechanism for the routing caches.

Instead of using read spin locks, we now use RCU locks(read-copy-update). This will get big gain in the route cache lookup performance, when huge route lookups are performed on the SMP Linux machine.
ip_route_output_key() takes additional argument 'struct flowi' and so also ip_route_output_slow(). In 2.4.x 'struct rt_key' is used instead for searching. struct flowi is more comprehensive search key to find more perfect match for a given flow for e.g., sourrce/destination port numbers.
This is one of the major difference but updting base routing table FIB, has not changed and has no performance gain in searching/inserting/deleting entry from here.
Another area where I see major difference is the socket data-structure. Once again the changes have got nothing much to do with performance. It is just better organization of the socket structure for better code readability. Following macros are used to access protocol specific socket information from 'struct sock' -
inet_sk() for getting pointer to inet protocol family.
udp/tcp_sk() for getting udp/tcp protocol specific socket pointer. This will map original 'struct sock' to struct tcp/udp_sock.
Entire description can't go here as it requires big space. But, I'll keep updating the entries, whereever I find major differences that can be explained in short. If we know the baisc skeleton and terminology of the Linux networking kernel, it will be easy to understand the most recent code and reason it out.
I'm working on next revision where I'm putting all the details related to the differences and much more........!!!!

Friday, February 13, 2009

book piracy and terrorism

I find myself in such a helpless situation, when I get to see sites allowing free downloads of the recently published titles. Some of the countries exploit loopholes in DMCA and allow uploads of complete book content which have not yet soaked in the market. There are much better ways to share knowledge instead of uploading scanned books. People who can not afford to buy the book can always make use of libraries or can share a book. But downloading free books from the sites, is almost supporting crime. First of all, it is the moral responsibility of the internet site owners to avoid such illegal practice in the name of "sharing knowledge". If this is for social cause, they should open a library and buy in books for everyone. Sharing knowledge agenda can never go together with theft and illegal practices. Yes, I'm talking about http://www.rapidshare.com and all other such sites who are responsible for all such illegal practices. I appeal to the respectable audience to avoid downloading books from these sites. Instead use libraries or share the book with others but avoid supporting illegal practices. There is effort and resources involved both from author and publishers end. Publishing companies and book stores are getting income by selling these books. Sales of a copy of the title will earn bread for so many in the chain. Lastly, knowledge gained by any illegal means will never solve any purpose. I hope for co-operation from at least those who are downloading books. If they avoid downloading books from such illegal sites, it will surely discourage site owners from carrying such illegal and immoral practices.

Monday, February 2, 2009

about my book Linux TCP/IP - ISBN 0470147733

Hi, This is the first time I'm blogging on the site so would need some introduction. I introduce my self as Sameer Seth who has been associated with the software industry for around a decade. I worked with Sun Microsystems as a internet engineer, where I worked on NFS, TCP/IP stack and streams framework. Currently working with Juniper as a senior engineer in the junos kernel team. My activities revolve mostly around opensource comminuty mostly to serve it in whatever way I can. I've worked for opensolaris and Linux is anytime favourite as I've known the industry by this name since I entered into it. As far as opensolaris is concerned, my contribution are presentation on OpenSolaris internet technology and blogging goes here - http://blogs.sun.com/sameers I've recently written a book along with my co-author Ajay with a different approach that explains "TCP/IP Architecture, Design and Implementation in Linux" - 0470147733. Amazon link to the title "TCP/IP Architecture, Design and Implementation in Linux" is - http://www.amazon.com/exec/obidos/ASIN/0470147733/ref=nosim/porfessionalp4-20 conceptualization of "TCP/IP Architecture, Design and Implementation in Linux" The book was started long back when kernel 2.4 was most stable and acceptable with point of view of just ducomenting TCP/IP internals in Linux but never thought it would take the shape of the book. When the title was accepted by the publisher(IEEE Computer-society - Wiley), kernel 2.6 had already appeared and was being accepted slowly. That was the time when decision was to be made whether to make it for kernel 2.6. The time was little to do all this because the format of the book and descriptions was such that huge amount of work was required to convert the existing work. The reason was that code snippets included line numbers and explanations also had the line numbers in the description. So, we planned to keep it for kernel 2.4 and to revise the contents for kernel 2.6 in the next edition. what is different in "TCP/IP Architecture, Design and Implementation in Linux" This was all about the 2.4 and 2.6 fight. The intent of writing a book was not show the latest but to bridge the gap. There are good books that explain Linux network internals and one of those is "Lnux networking Internals" by Christian Benvenuti. It explains IP and related protocols to a great extent. But when I am a new comer to Linux networking stack, I'd like to know very systematically what is happening when - we do recv/send over the TCP socket ? - connection is established? - what does socket, bind, listen, accept, connect system calls do internally - how is route look up done, - how is packet transmitted and received end to end in the linux kernel - how is TCP timers, memory managed. - how is TCP state machine implemented and congestion handled.
- QOS implementation and how does it fit in to the stack with implementation details illustrating CBQ.
- how are various layers in the stack linked and the boundaries for each layer.
- how is the net filter hooks implemented and their relevance in the stack.
- Implementation details for ipchain and iptables.
- NET softIRQ implementation and it's relevance with respect to packet processing on SMP architecture. I did not find any book tying up these ends together and linking each part of the kernel responsible for making the stack. I did not find exact explanation and working of net softIRQ and even if someone explains, it's relevance with respect to packet reception and transmission and scalability issues on SMP architecture is not very well explained. Moreover, no book explains TCP and it's implementaiton for Linux. All together with net-filters(network hooks in the Linux stack framework) and QOS implementaion is not provided by any existing book in one place. For example, net-filter framework will give a very clear picture of what needs to be done when you want to implement your own extensions to the TCP/IP stack like implementing NAT, masquerading but it will not explain details of NAT and masquerading. So, I though of bridging these gaps along with the explanations for all those topics which have already been covered in details by other authors in my own way. The intent was not to compete with the good existing titles, it was to bring something which doesn't exist. Why not to bother about 2.4 and 2.6 with "TCP/IP Architecture, Design and Implementation in Linux" Many question me - why not for kernel 2.6? So I'd like to say that if you are a learner, kernel version should not matter much(2.4<->2.6). These are subsequent versions and differences will surely be there but basic skeleton will never change. Most of the important changes as far as networking stack and supporting framework is concerned, was done from 2.2->2.4(soft IRQ). Rest what we see is new features being added on top of the existing skeleton, data-structure layout changes to accomodate those changes or for some code optimization or for performance enhancements. Some of the changes will be clubbing of two or more functions to create a new function. These changes will matter for those who are contributing to kernel 2.6 directly. For those who are starting their journey into Linux networking stack(even for those new comers who are directly touching kernel 2.6), 2.4 or 2.6 will not matter as far as networking stack is concerned. If you are a starter and you don't have any idea, getting knowledge of kernel 2.4 implementation will surely help diving into kernel 2.6. All you need to know is - what to look where - which part is responsible for what in the TCP/IP stack - what all frameworks are supporting it - how are these various parts linked. If you know most of the things, still this book can work as reference point for most of the things. debugging related chapter in "TCP/IP Architecture, Design and Implementation in Linux" As far as debugging chapter is concerned, lcrash may be outdated for kernel 2.6 but I'd say it is still supported by kernel 2.6. The intent here was not to show what lcrash or other debuggers can do, but to let the readers visualise what all kernel data-structures are involved and to intersect the stack when different operations are performed. There will be equivalent commands supported by different debuggers as offered by lcrash that can be used to do the same stuff as shown in the debugging chapter. So, this shoulld not matter. Expectations from "TCP/IP Architecture, Design and Implementation in Linux" So, the main attraction of the book is the systematic approach to understanding Linux TCP/IP stack implemantation and other supporting kernel framework. The intent is to bridge gaps from the existing titles and not to re-work what others have done beautifully. This will not only help new enthusiasts to have better understandng about the subject but also act as reference point for the contributers to linux community with experience. I've started planning about the next edition that explains latest kernel 2.6. But that is a long way to go and I make sure that it should appear before next kernel version is out, surely. I hope this information will be helpful. If anyone is interested in reviewing the book, please let me know along with brief biodata about youself(required by the publisher). In case some one is interested and qualifies too, a copy will be sent on the mentioned address.