What are the differences between Windows and Linux?

There are many differences between Windows NT and Linux. Skipping over the obvious ones ( Windows NT is made by Microsoft, Linux has a higher server market share, etc...), probably the most interesting difference is how security is structured. Linux's basic security / permissions structure is modeled after that of older Unices. Different users are assigned different privileges, and a single "root" user can do anything. Windows NT has no "root" user, and no single account has absolute power. In theory, this makes NT more secure, since more accounts would have to be compromised to gain control of the system. In practice, a single "Administrator" account is all that is usually needed to hijack an NT system, and many Linux distributions disable the root account by default.

Windows was introduced by Microsoft in 1983, and has been the dominant Operating System available for the PC since the early-nineties. As such, Microsoft has enjoyed great financial success, and Windows has had many years and incredible fiscal resources to evolve to meet the demands of the mass-market. There is an staggeringly rich set of features here, from very explicit, step-by-step user interfaces for the first time computer user, to powerful interfaces for the computer professional, and everything in between. By contrast, Linux achieved notoriety a bit later, in the mid-nineties, with a distribution known as Redhat, and although Linux was built on more mature, stable underpinnings (Unix), it did not enjoy nearly the same marketing or development budget that Microsoft threw behind Windows. In fact, the developers of Linux are commonly credited as founding the Open Source Software movement, which is the idea that software can be made better through the free sharing of its source code. In this philosophy, programmers often volunteer their time to develop software for free, as was done with Linux, and Linux is still available for free in its more basic forms. Companies like Redhat only make money by "packaging" Linux with printed documentation, extra software utilities, and setup wizards designed to make the installation of Linux and its subsequent software packages easier. Even so, the amount of money they are able to generate this way is paltry compared to the wealth of Microsoft (which makes most company's financials look paltry). Because of this, the marketing behind Linux has been miniscule compared to that of Windows, and its lack of acceptance among less technical users reflects this. A large reason is because Windows has established a very deeply-ingrained (and some have argued unfairly controlling) relationship with PC hardware manufacturers, ensuring that almost every new PC ships with Windows installed from day one. Given that they must satisfy the demands (though perhaps less than perfectly) of the majority of novice computer users, and add to that the amount of time and money that Windows has enjoyed to make itself accessible to these users, and it is easy to see why Windows is generally regarded as superior to Linux in the area of accessibility to novices. There are graphic user interfaces (abbreviated as GUI) present for almost everything you could want to do, and there is almost always more than one way to do it. In fact, one common criticism of Windows is that so many features have been layered on top of one another over the years, that it has become an overly-complicated, almost labyrinthine user experience. By contrast, some may find Linux to be more streamlined; however, there are still many equivalent features in Windows for which Linux does not provide a GUI, and the user is forced to type textual instructions into a command-line interface, or shell. While many power users consider this a plus, it is unrealistic to demand this of novice computer users, and novice users should bear this strongly in mind. All this being said, Linux still shines brilliantly in some areas that Windows seems to consistently flounder. Because the underlying architecture of Linux is more mature, stable, and secure than Windows, Linux "crashes" and "freezes" significantly less often, and can run continuously without problems for months or even years without being "rebooted". In addition, Linux does not suffer from the same security flaws as Windows, and your chances of contracting a virus, a worm, or some other form of predatory software is much lower. On an more subjective note, I suspect that given the same time and monetary advantages as Windows, Linux might have easily developed into a superior operating system in every regard.
As it stands today, they each have pros and cons. Windows is widely accepted everywhere, boasts an enormous plethora of GUIs, and has millions of software packages that run under it. But it is buggier, less secure, and sometimes feels cavernous. Linux is solid and smooth running, and feels more stream-lined to many. But what technical users call stream-lined, novices may interpret as spare, and sometimes barren or just plain missin. There are also fewer software packages available for it currently, though many of those that are available are free. As time goes by, and the Open Software Community develops more for Linux, these differences will shrink, but until there is financially powerful, unifying force (company) behind Linux, this author thinks it is doubtful they will go away all together.

1. Linux is a open-source OS.People can change code and add programs which will help to use your computer better. It's designed as a reaction on the monopoly position of windows. you can't change any thing in windows. you can't even see which processes do what and build your onw extension. Linux wants the programmers to extend and redesign it's OS time after time, so it beats Windows or at least is as good as windows, but whit open-source, so you can see what happens and you can edit the OS.
   
2. All the flavors of Windows come from Microsoft, the various distributions of Linux come from different companies (i.e LIndows , Lycoris, Red Hat, SuSe, Mandrake, Knopping, Slackware).


3. Linux is customizable in a way that Windows is not. For example,NASlite is a version of Linux that runs off a single floppy disk and converts an old computer into a file server. This ultra small edition of Linux is capable of networking, file sharing and being a web server.


4. For desktop or home use, Linux is very cheap or free, Windows is expensive. For server use, Linux is very cheap compared to Windows. Microsoft allows a single copy of Windows to be used on only one computer. Starting with Windows XP, they use software to enforce this rule (activation). In contrast, once you have purchased Linux, you can run it on any number of computers for no additional charge.


5. You have to log on to Linux with a userid and password. This is not true of Windows. Typically Windows 9x does not ask for a userid/password at boot time and, even if it does, this can be easily bypassed. In general, Windows NT, 2000 and XP do require a userid/password to log on. However Windows 2000 and XP can be configured with a default userid and password so they boot directly to the Windows desktop. Windows XP, 2000 and Linux all support different types of users. Windows XP Home Edition supports Administrator class users that have full and total access to the system and restricted users that, among other restrictions, can't install software. Windows XP Pro and Windows 2000 support additional levels of users and there are groups of system privileges that can be assigned to a particular user. In Linux, the user with full and total access is called root, everyone else is a normal user. The options for Linux security privileges don't seem to me to be as robust as in Windows 2000 and XP Pro, they are focused on files and directories (can you read, update and execute files). Linux has a concept of a group of users that Windows does not, but again the privileges associated with a group are all file/directory related.


6. Linux has a reputation for fewer bugs than Windows.


7. Windows must boot from a primary partition. Linux can boot from either a primary partition or a logical partition inside an extended partition. Windows must boot from the first hard disk. Linux can boot from any hard disk in the computer.


8. Windows uses a hidden file for its swap file. Typically this file resides in the same partition as the OS (advanced users can opt to put the file in another partition). Linux uses a dedicated partition for its swap file (advanced users can opt to implement the swap file as a file in the same partition as the OS).


9. Windows uses FAT12, FAT16, FAT32 and/or NTFS with NTFS almost always being the best choice. Linux also has a number of its own native file systems. The default file systeAll the file systems use directories and subdirectories. Windows separates directories with a back slash, Linux uses a normal forward slash. Windows file names are not case sensitive. Linux file names are. For example "abc" and "aBC" are different files in Linux, whereas in Windows it would refer to the same file. Case sensitivity has been a problem for this very web page, the name of which is "Linux.vs.Windows.html". At times, people have tried to get to this page using "linux.vs.windows.html" (all lower case) which resulted in a Page Not Found error. Eventually, I created a new web page with the name in all lower case and this new page simply re-directs you to the real page, the one you are reading now (with a capital L and W). m for Linux used to be ext2, now it is typically ext3.


10. Windows and Linux use different concepts for their file hierarchy. Windows uses a volume-based file hierarchy, Linux uses a unified scheme. Windows uses letters of the alphabet to represent different devices and different hard disk partitions. Under Windows, you need to know what volume (C:, D:,...) a file resides on to select it, the file's physical location is part of it's name. In Linux all directories are attached to the root directory, which is identified by a forward-slash, "/". For example, below are some second-level directories: /bin/ ---- system binaries, user programs with normal user permissions /sbin --- executables that need root permission /data/ --- a user defined directory /dev/ ---- system device tree /etc/ ---- system configuration /home/ --- users' subdirectories /home/{username} akin to the Windows My Documents folder /tmp/ ---- system temporary files /usr/ ---- applications software /usr/bin - executables for programs with user permission /var/ ---- system variables /lib --- libraries needed for installed programs to run .


11. Both support the concept of hidden files, which are files that, by default, are not shown to the user when listing files in a directory. Linux implements this with a filename that starts with a period. Windows tracks this as a file attribute in the file metadata (along with things like the last update date). In both OSs the user can over-ride the default behavior and force the system to list hidden files.


12. Windows started with BAT files (a combination of OS commands and optionally its own language) and then progressed to Windows Scripting Host (WSH) which supports two languages, JavaScript and VB Script. Linux, like all Unix variants, provides multiple scripting languages, referred to as shell scripts. In general, the Linux scripting languages are older and cruder than WSH but much more powerful than BAT files. They tend to use special characters instead of English commands and don't support objects (this only matters to programmers). One scripting language that can run on both Linux and Windows is PHP. It always has to be installed under Windows, it may have to be installed under Linux. PHP is typically found running on Linux based web servers in combination with Apache, but it is capable of running "client side" (on your computer).


13. Every computer printer ships with drivers for last last few versions of Windows (at the time it was manufactured). Running the printer on a very old or too new version of Windows may or may not work. Still, this a far better situation than with Linux which does not support as many printers as Windows. In an environment with many Linux users, shared network printers a tech support staff, this should not be an issue as you can limit yourself to well supported printers. Home users of Linux however, will no doubt suffer from the relatively poor support for printers.


14. Windows allows programs to store user information (files and settings) anywhere. This makes it impossibly hard to backup user data files and settings and to switch to a new computer. In contrast, Linux stores all user data in the home directory making it much easier to migrate from an old computer to a new one. If home directories are segregated in their own partition, you can even upgrade from one version of Linux to another without having to migrate user data and settings.

Linux and NT Server 4.0

Since NT is often chosen on the basis of cost-effective hardware solutions, Linux will be the UNIX system in this comparison, for it thrives on Intel hardware. Note: Only the items/features that actually ship with each operating system are listed here. Perl 5.0, for instance, is available for all platforms, but Microsoft does not provide this with its operating systems. On the same note, most distributions of Linux ship with only about four GUIs (window managers) to choose from, yet you'll note from a previous section in this article, that this is only a small number of what is available for Linux, or any other UNIX operating system for that matter.

Windows vs. Linux Design
It is possible that email and browser-based viruses, Trojans and worms are the source of the myth that Windows is attacked more often than Linux. Clearly there are more desktop installations of Windows than Linux. It is certainly possible, if not probable, that Windows desktop software is attacked more often because Windows dominates the desktop. But this leaves an important question unanswered. Do the attacks so often succeed on Windows because the attacks are so numerous, or because there are inherent design flaws and poor design decisions in Windows?
Many, if not most of the viruses, Trojans, worms and other malware that infect Windows machines do so through vulnerabilities in Microsoft Outlook and Internet Explorer. To put the question another way, given the same type of desktop software on Linux (the most often used web browsers, email, word processors, etc.), Are there as many security vulnerabilities on Linux as Windows?

Windows Design
Viruses, Trojans and other malware make it onto Windows desktops for a number of reasons familiar to Windows and foreign to Linux:
1. Windows has only recently evolved from a single-user design to a multi-user model
2. Windows is monolithic, not modular, by design
3. Windows depends too heavily on an RPC model
4. Windows focuses on its familiar graphical desktop interface
Windows has only recently evolved from a single-user design to a multi-user model
Critics of Linux are fond of saying that Linux is “old” technology. Ironically, one of the biggest problems with Windows is that it hasn’t been able to escape its “old” legacy single-user design. Windows has long been hampered by its origin as a single-user system. Windows was originally designed to allow both users and applications free access to the entire system, which means anyone could tamper with a critical system program or file. It also means viruses, Trojans and other malware could tamper with any critical system program or file, because Windows did not isolate users or applications from these sensitive areas of the operating system.
Windows XP was the first version of Windows to reflect a serious effort to isolate users from the system, so that users each have their own private files and limited system privileges. This caused many legacy Windows applications to fail, because they were used to being able to access and modify programs and files that only an administrator should be able to access. That’s why Windows XP includes a compatibility mode - a mode that allows programs to operate as if they were running in the original insecure single-user design. This is also why each new version of Windows threatens to break applications that ran on previous versions. As Microsoft is forced to hack Windows into behaving more like a multi-usersystem, the new restrictions break applications that are used to working without those restraints.
Windows XP represented progress, but even Windows XP could not be justifiably referred to as a true multi-user system. For example, Windows XP supports what Microsoft calls “Fast User Switching”, which means that two or more people can log into a Windows XP system on a single PC at the same time. Here’s the catch. This is only possible if and only if the PC is not set up to be part of a Windows network domain. That’s because Microsoft networking was designed under the assumption that people who log into a network will do so from their own PC. Microsoft was either unable or unwilling to make the necessary changes to the operating system and network design to accommodate this scenario for Windows XP.
Windows Server 2003 makes some more progress toward true multi-user capabilities, but even Windows Server 2003 hasn’t escaped all of the leftover single-user security holes. That’s why Windows Server 2003 has to turn off many browser capabilities (such as ActiveX, scripting, etc.) by default. If Microsoft had redesigned these features to work in a safe, isolated manner within a true multi-user environment, these features would not present the severe risks that continue to plague Windows.

Linux Design
According to the Summer 2004 Evans Data Linux Developers Survey, 93% of Linux developers have experienced two or fewer incidents where a Linux machine was compromised. Eighty-seven percent had experienced only one such incident, and 78% have never had a cracker break into a Linux machine. In the few cases where intruders succeeded, the primary cause was inadequately configured security settings.
More relevant to this discussion, however, is the fact that 92% of those surveyed have never experienced a virus, Trojan, or other malware infection on Linux.
Viruses, Trojans and other malware rarely, if ever, manage to infect Linux systems, in part because:
1. Linux is based on a long history of well fleshed-out multi-user design
2. Linux is mostly modular by design
3. Linux does not depend upon RPC to function, and services are usually configured not to use RPC by default
4. Linux servers are ideal for headless non-local administration
Keep in mind when reading the summaries below that there are variations in the default configurations of the different distributions of Linux, so what may be true of Red Hat Linux may not be true of Debian and there may be even more differences in SuSE. For the most part, all the major Linux distributions tend to follow sane guidelines in the default configurations.

Linux is based on a long history of well fleshed-out multi-user design
Linux does not have a history of being a single-user system. Therefore it has been designed from the ground-up to isolate users from applications, files and directories that affect the entire operating system. Each user is given a user directory where all of the user’s data files and configuration files are stored. When a user runs an application, such as a word processor, that word processor runs with the restricted privileges of the user. It can only write to the user’s own home directory. It cannot write to a system file or even to another user’s directory unless the administrator explicitly gives the user permission to do so.
Even more important, Linux provides almost all capabilities, such as the rendering of JPEG images, as modular libraries. As a result, when a word processor renders JPEG images, the JPEG rendering functions will run with the same restricted privileges as the word processor itself. If there is a flaw in the JPEG rendering routines, a malicious hacker can only exploit this flaw to gain the same privileges as the user, thus limiting the potential damage. This is the benefit of a modular system, and it follows more closely the spherical analogy of an ideally designed operating system (see the section Windows is Monolithic by Design, not Modular).
Given the default restrictions in the modular nature of Linux; it is nearly impossible to send an email to a Linux user that will infect the entire machine with a virus. It doesn’t matter how poorly the email client is designed or how badly it may behave - it only has the privileges to infect or damage the user’s own files. Linux browsers do not support inherently insecure objects such as ActiveX controls, but even if they did, a malicious ActiveX control would only run with the privileges of the user who is running the browser. Once again, the most damage it could do is infect or delete the user’s own files.
Even services, such as web servers, typically run as users with restricted privileges. For example, Debian GNU/Linux runs the Apache server as the user www-data, who belongs to a group with the same name, www-data. If a malicious hacker manages to gain complete control over the Apache web server on a Debian system, that hacker can only affect files owned by the user www-data, such as web pages. In turn, the MySQL SQL database server often used in conjunction with Apache, runs with the privileges of the user mysql. So even if Apache and MySQL are used together to serve web pages, a malicious hacker who gains control of Apache does not have the privileges to exploit the Apache hole in order to gain control of the database server, because the database server is “owned” by another user.
In addition, users associated with services such as Apache, MySQL, etc., are often set up with user accounts that have no access to a command line. So if a malicious hacker somehow breaks into the MySQL user account, that hacker cannot exploit that vulnerability to issue arbitrary commands to the Linux server, because that account has no ability to issue commands.
In sharp contrast, Windows was originally designed to allow all users and applications to have administrator access to every file on the system. Windows has only gradually been re-worked to isolate users and what they do from the rest of the system. Windows Server 2003 is close to achieving this goal, but the methodology Microsoft has employed to create this barrier between user and system is still largely composed of constantly changing hacks to the existing design, rather than a fundamental redesign with multi-user capability and security as the foundational concept behind the system.

Problems and Their Solution

PROBLEM 1

PROBLEM: I have install windows Service Pack 2 and after updating windows upto Service Pack 3 , I am able to login system, but receiving a continuous message that "It is not a Genuine copy of Window". What are the solutions available to this problem?

SOLUTION:

Microsoft, in a rather brilliant move, has released an update that checks your OS serial against a list of known leaked serials (corporate edition keys, etc). If your serial matches one of them then it puts a darling little message on the bottom right hand of your login screen, and also a system tray icon that nags you to buy a genuine copy of windows.
It will also pop up a dialog that says “This copy of Windows is not Genuine” “This copy of Windows is not genuine and you have not yet resolved this issue. This computer is no longer eligible to recieve select security upgrades from Microsoft.
To protect your copy of Windows, you must click Get Genuine now.”
Every so often a little balloon will pop up that says “You may be a victim of software counterfeiting.” Below that it will say “This copy of Windows is not genuine. Click this balloon to resolve now.” If you’re working on a system that has this running it gets annoying in a hurry. What the time interval between pop-ups is, I don’t know, but I imagine it’ll drive most people to buy a legit key after a while just to get the thing to shut up.
Fixing the popup is easy enough.

• Reboot to safe mode. Open up HijackThis and scroll down to the selection that says “020 – Winlogon Nofity: Wgalogon – …” select that, and click fix checked.


• You can also go and delete the actual executable that nags you. The full path to it should be (for most installs) C:\Windows\System32\wgatray.exe.


• Reboot after this. No longer should you get the popups and that nagger as you log in should not show up either.
  

If you have a valid copy of Windows - You can do the following:

1. Launch Windows Task Manager.
2. End wgatray.exe process in Task Manager.
3. Restart Windows XP in Safe Mode.
4. Delete WgaTray.exe from c:\Windows\System32.
5. Delete WgaTray.exe from c:\Windows\System32\dllcache.
6. Launch RegEdit.
7. Browse to the following location : HKEY_LOCAL_MACHINE \ SOFTWARE \ Microsoft \ Windows NT \ CurrentVersion \ Winlogon\Notify
8. Delete the folder "WgaLogon" and all its contents
9. Reboot Windows XP.

In Brief :

1. Open system32 - Start/Run/system32 (ENTER)
2. Go to wgaTray.exe
3. Push Delete on your keyboard (but do not click yes or no)
4. Open window task manager (Ctrl+Alt+Del)
5. Click processes
6. Go to wgatray.exe (press Delete button)
7. Now line up the 2 windows that say "yes & no" and then click yes to end the process & also yes to delete in this order

PROBLEM - 2

PROBLEM : You have downloaded windows 7 from Microsoft Official Website in December 2009. On present day, your system is rebooting after 2 hours. What are the solutions available to overcome this problem ?

SOLUTION :
Windows Communications Manager at Microsoft, posted the news on The Windows Blog stating:

For the RC, bi-hourly shutdowns will begin on March 1st, 2010. You will be alerted to install a released version of Windows and your PC will shut down automatically every 2 hours. On June 1st, 2010 if you are still on the Windows 7 RC your license for the Windows 7 RC will expire and the non-genuine experience is triggered where your wallpaper is removed and “This copy of Windows is not genuine” will be displayed in the lower right corner above the taskbar.

This isn’t a new tactic Microsoft has implemented to remind users they need to upgrade and it did the same thing with Vista previews. Windows 7 is expected to release in October this year, but at the very latest will be out by January next year giving you plenty of time to buy a copy before the automatic shutdowns begin.

Windows 7 RC to Have Shutdowns on March 1 '10

Last week the news made the rounds of the Windows 7 RC having a longer than average trial period. Stated clearly by Microsoft is that the Release Candidate install will remain valid until June 1, 2010.
What wasn’t so clearly publicized is the small detail that starting March 1, 2010 there will be bihourly shut downs of any installs of Windows 7 RC.
As Microsoft details on its partner site regarding the shutdown schedules for the beta and RC:

To avoid interruption, it’s recommended that you and your customers rebuild test machines by using a valid Windows operating system before Windows 7 Beta and Windows 7 RC expire. Windows will automatically notify you that the expiration process is about to begin. Two weeks later, your PC will shut down every two hours. For Windows 7 Beta, the bihourly shutdowns will begin July 1, 2009. The software will expire August 1, 2009. For Windows 7 RC, the bihourly shutdowns will begin March 1, 2010. The software will expire June 1, 2010.
So there you have it. While Windows 7 RC will be useful for 13 months as we all thought last week, it’ll be more useful for the first 10 months.
If there’s any confusion regarding what bihourly means – if it’ll be shutdowns every 30 minutes or 2 hours – it’s the longer of the two options, though probably still annoying enough to make one want to upgrade to the final version.
Windows 7 RC is set for a public launch tomorrow. Look forward to it! (Unless you’ve already downloaded it.)
How to stop Windows 7 reboot loop
We know pretty well that Windows 7 is a quite new operating system which is still in its RC version and therefore not ready (at the time writing) for the market and regular not-savvy users. That’s why we should not be surprise if, every now and then it reboots automatically to prevent the computer from being damaged beyond recovery. Such a behaviour is preset by Microsoft and it is triggered upon system failure. Anyway, if you are not a lucky person your Windows 7 may become stuck and therefore enters in a infinite reboot loop, an endless series of turn-off and turn-on loops without any apparent way to get out of it. Let’s s lean how to stop Windows 7 reboot loop thanks to a little trick!

1. When your computer starts its next reboot(black screen) , just hit F8 in your keyboard a lot of times till you get the Advanced Boot Options screen.
   
2. Now, with your arrow keys, select Disable automatic restart on system failure and press Enter.


3. Well Done.

NTFS vs FAT File System

NTFS

NTFS is a high-performance and self-healing file system proprietary to Windows XP Vista 2003,2000, NT & Windows 7, which supports file-level security, compression and auditing. It also supports large volumes and powerful storage solution such as RAID. The most important new feature of NTFS is the ability to encrypt files and folders to protect your sensitive data.

NTFS is a Microsoft file system. It was introduced in Windows NT and has been the default file system for every version of Microsoft Windows since. NTFS replaced the aged FAT file system and addresses most of FAT's shortcomings. NTFS has been continuously maintained and improved by Microsoft, and the current version provides secure data storage that meets the requirements of modern hardware and usage. However, NTFS remains a closed standard, Microsoft does not publish its API nor implementation details. Therefore only Microsoft operating systems can use NTFS natively, and even OS's that are capable of reading and writing to NTFS cannot be installed on hard disks formatted as NTFS.

The NTFS acronym stands for New Technology File System. The name derives from the implementation of very innovative data storage techniques that were refined in NTFS. While none of the techniques are unique to NTFS, it is the first time that so many innovations were released at once on a production file system. The FAT file system had long been criticized for not including some of the more obvious improvements such as journaling, disk quotas, and file compression. However, these improvements made NTFS incompatible with previous versions of Windows, and also with hard disk tools designed for FAT file systems. For example, data recovery tools such as GetDataBack and partitioning tools such as PartitionMagic would run on Windows NT, yet could not function on the newer file system. This led to much frustration with users who had purchased licenses for these products before upgrading to Windows NT.

NTFS Master File Table (MFT)
Each file on an NTFS volume is represented by a record in a special file called the master file table (MFT). NTFS reserves the first 16 records of the table for special information. The first record of this table describes the master file table itself, followed by a MFT mirror record. If the first MFT record is corrupted, NTFS reads the second record to find the MFT mirror file, whose first record is identical to the first record of the MFT. The locations of the data segments for both the MFT and MFT mirror file are recorded in the boot sector. A duplicate of the boot sector is located at the logical center of the disk.
The third record of the MFT is the log file, used for file recovery. The seventeenth and following records of the master file table are for each file and directory (also viewed as a file by NTFS) on the volume.

Metafiles
The first 16 NTFS files (metafiles) are system files. Each of them is responsible for some aspect of system operation. The advantage of such modular approach is in amazing flexibility - for example on FAT the physical failure in the FAT area is fatal for all disk operation. As for NTFS it can displace and even fragment on the disk all system areas avoiding any damage of the surface except the first 16 MFT elements.
The metafiles are in the NTFS disk root directory, they start with a name character "$", though it is difficult to get any information about them by standard means. Curiously that even for these files the quite real size is reported, and it is possible to find out for example how many operating system spends on cataloguing of all your disk having looked at $MFT file size. In the following table the metafiles used at the moment and their function are indicated.

Files and streams
So the system has files and nothing except files. What does this concept on NTFS include?

First of all the compulsory element is the record in MFT. As it was said above all disk files are mentioned in MFT. All information about a file except data itself is stored in this place: a file name, its size, separate fragments position on the disk, etc. If one MFT record is not enough for information, then several records are used and not obligatory one after another. Optional element is file data streams. The definition "optional" seems to be a bit strange but nevertheless there is nothing strange here. Firstly a file may not have data and in this case disk free space isn't used on it. Secondly a file may have not very big size. Then a rather successful decision is applied: file data are stored just in MFT, in the place free from the master data in limits of one MFT record. The files with the size of hundreds byte usually don't have "physical" image in the fundamental file area. All such file data are stored in one place - in MFT.

The directories
The directory on NTFS is a specific file storing the references to other files and directories establishing the hierarchical constitution of disk data. The directory file is divided into blocks, each of them contains a file name, base attributes and reference to the element MFT which already gives the complete information on an element of the directory. The inner structure of the directory is a binary tree. It means that to search the file with the given name in the linear directory such for example as for FAT, the operating system should look through all elements of the directory until it finds the necessary one. The binary tree allocates the names of files to make the file search faster - with the help of obtaining binary answers to the questions about the file position. The binary tree is capable to give the answer to the question in what group the required name is situated - above or below the given element. We begin with such question to the average element, and each answer narrows the area of search twice. The files are sorted according to the alphabet, and the answer to the question is carried out by the obvious way - matching of initial letters. The search area which has been narrowed twice starts to be researched the same way starting again from the average element.

FAT

The FAT file system was first introduced in the days of MS-DOS way back in 1981. The purpose of the File Allocation Table is to provide the mapping between clusters - the basic unit of logical storage on a disk at the operating system level - and the physical location of data in terms of cylinders, tracks and sectors - the form of addressing used by the drive's hardware controller.
The FAT contains an entry for every file stored on the volume that contains the address of the file's starting cluster. Each cluster contains a pointer to the next cluster in the file, or an end-of-file indicator at (0xFFFF), which indicates that this cluster is the end of the file. The diagram shows three files: File1.txt uses three clusters, File2.txt is a fragmented file that requires three clusters and File3.txt fits in one cluster. In each case, the file allocation table entry points to the first cluster of the file.

The FAT16 File System
The FAT16 file system is compatible with the majority of operating systems. This is evident by MS DOS, Windows 95, Windows 98, Windows Me, Windows NT, Windows 2000 and Windows XP being able to utilize the FAT16 file system. FAT16 generally works well in managing disk space when the size of the volume is less than 256MB. You should refrain from using FAT16 on volumes that are larger than 512MB. FAT16 cannot be utilized on volumes that exceed 4 GB.
FAT16 maps clusters on the FAT partition. A cluster is the smallest unit that the OS operating system utilizes when it assigns space on the partition. A cluster is also at times referred to as an allocation unit.

The file allocation table identifies a cluster in the FAT partition as either:
· Unused
· Cluster in use by a file
· Bad cluster
· Last cluster in a file
The FAT16 volume is structured as follows:
· Boot sector on the system partition
· The primary file allocation table
· The copy or duplicate file allocation table
· A root folder
· Other folders and all files

The root folder holds an entry for each file and folder stored on the FAT16 volume and has its maximum number of table entries set at 512 for each disk drive. A file's or folder's entry contains the information listed below:
· Name: This is in 8.3 format
· Attribute: 8 bits
· Create time: 24 bits
· Create date: 16 bits
· Last access date: 16 bits
· Last modified time: 16 bits
· Last modified date: 16 bits
· Starting cluster number in the file allocation table: 16 bits
· File size: 32 bits
Disavantages of FAT16

A few disadvantages associated with the FAT16 file system are summarized below:
· The FAT16 file system has no local security for the file system or compression features.
· The boot sector is not backed up.
· The root folder can only have a maximum of 512 entries which means that files which have long names can greatly decrease the number of entries available.
· FAT16 does not work well with volume sizes that are large.

The FAT32 File System
The FAT32 file system can handle larger partitions than what the FAT16 file system can handle. FAT32 can support partitions up to 2047 GB in size compared to FAT16's 4 GB. With FAT32, there is no restriction on the number of entries that the root folder can contain. With FAT16, the root folder could only contain a maximum of 512 entries. The boot sector is also backed up on FAT32 volumes. A FAT32 volume must however have a minimum of 65,527 clusters.
The FAT32 architecture is very much like the architecture of the FAT16 file system. FAT32 was designed with little architectural changes to ensure compatibility with existing programs and device drivers. What this means is that device drivers and FAT tools used for FAT16 partitions would continue to work for FAT32 partitions.
FAT32 does however need 4 bytes in the file allocation table to store cluster values. This has led to the revision or expansion of internal data structures, on-disk data structures and published APIs.
A few disadvantages associated with the FAT32 file system are summarized below:
· Like the FAT16 file system, the FAT32 file system includes no local security for the files system or compression features.
· The MS-DOS, Windows 95, and Windows NT 4.0 OSs are unable to access or read FAT32 partitions.
· Both FAT16 and FAT32 partitions do not scale well - the file allocation table increases in size as the volume grows.

NTFS vs FAT

NTFS vs FAT32

Memory Management

MEMORY MANAGEMENT

The memory management subsystem is one of the most important parts of the operating system. Since the early days of computing, there has been a need for more memory than exists physically in a system. Strategies have been developed to overcome this limitation and the most successful of these is virtual memory. Virtual memory makes the system appear to have more memory than it actually has by sharing it between competing processes as they need it.
Virtual memory does more than just make your computer's memory go further. The memory management subsystem provides:

Large Address Spaces
The operating system makes the system appear as if it has a larger amount of memory than it actually has. The virtual memory can be many times larger than the physical memory in the system,
Protection
Each process in the system has its own virtual address space. These virtual address spaces are completely separate from each other and so a process running one application cannot affect another. Also, the hardware virtual memory mechanisms allow areas of memory to be protected against writing. This protects code and data from being overwritten by rogue applications.
Memory Mapping
Memory mapping is used to map image and data files into a processes address space. In memory mapping, the contents of a file are linked directly into the virtual address space of a process.
Fair Physical Memory Allocation
The memory management subsystem allows each running process in the system a fair share of the physical memory of the system,
Shared Virtual Memory
Although virtual memory allows processes to have separate (virtual) address spaces, there are times when you need processes to share memory. For example there could be several processes in the system running the bash command shell. Rather than have several copies of bash, one in each processes virtual address space, it is better to have only one copy in physical memory and all of the processes running bash share it. Dynamic libraries are another common example of executing code shared between several processes.

Shared memory can also be used as an Inter Process Communication (IPC) mechanism, with two or more processes exchanging information via memory common to all of them. Linux supports the Unix TM System V shared memory IPC.

Demand Paging
As there is much less physical memory than virtual memory the operating system must be careful that it does not use the physical memory inefficiently. One way to save physical memory is to only load virtual pages that are currently being used by the executing program. For example, a database program may be run to query a database. In this case not all of the database needs to be loaded into memory, just those data records that are being examined. If the database query is a search query then it does not make sense to load the code from the database program that deals with adding new records. This technique of only loading virtual pages into memory as they are accessed is known as demand paging.
When a process attempts to access a virtual address that is not currently in memory the processor cannot find a page table entry for the virtual page referenced. There is no entry in process X's page table for virtual page frame number 2 and so if process X attempts to read from an address within virtual page frame number 2 the processor cannot translate the address into a physical one. At this point the processor notifies the operating system that a page fault has occurred.

Swapping
If a process needs to bring a virtual page into physical memory and there are no free physical pages available, the operating system must make room for this page by discarding another page from physical memory.
If the page to be discarded from physical memory came from an image or data file and has not been written to then the page does not need to be saved. Instead it can be discarded and if the process needs that page again it can be brought back into memory from the image or data file.
However, if the page has been modified, the operating system must preserve the contents of that page so that it can be accessed at a later time. This type of page is known as a dirty page and when it is removed from memory it is saved in a special sort of file called the swap file. Accesses to the swap file are very long relative to the speed of the processor and physical memory and the operating system must juggle the need to write pages to disk with the need to retain them in memory to be used again.

Physical and Virtual Addressing Modes

It does not make much sense for the operating system itself to run in virtual memory. This would be a nightmare situation where the operating system must maintain page tables for itself. Most multi-purpose processors support the notion of a physical address mode as well as a virtual address mode. Physical addressing mode requires no page tables and the processor does not attempt to perform any address translations in this mode. The Linux kernel is linked to run in physical address space.
The Alpha AXP processor does not have a special physical addressing mode. Instead, it divides up the memory space into several areas and designates two of them as physically mapped addresses. This kernel address space is known as KSEG address space and it encompasses all addresses upwards from 0xfffffc0000000000. In order to execute from code linked in KSEG (by definition, kernel code) or access data there, the code must be executing in kernel mode. The Linux kernel on Alpha is linked to execute from address 0xfffffc0000310000.

Access Control
The page table entries also contain access control information. As the processor is already using the page table entry to map a processes virtual address to a physical one, it can easily use the access control information to check that the process is not accessing memory in a way that it should not.

Caches

If you were to implement a system using the above theoretical model then it would work, but not particularly efficiently. Both operating system and processor designers try hard to extract more performance from the system. Apart from making the processors, memory and so on faster the best approach is to maintain caches of useful information and data that make some operations faster. Linux uses a number of memory management related caches:
Buffer Cache
The buffer cache contains data buffers that are used by the block device drivers.
These buffers are of fixed sizes (for example 512 bytes) and contain blocks of information that have either been read from a block device or are being written to it. The buffer cache is indexed via the device identifier and the desired block number and is used to quickly find a block of data. Block devices are only ever accessed via the buffer cache. If data can be found in the buffer cache then it does not need to be read from the physical block device, for example a hard disk, and access to it is much faster.
Page Cache
This is used to speed up access to images and data on disk.
It is used to cache the logical contents of a file a page at a time and is accessed via the file and offset within the file. As pages are read into memory from disk, they are cached in the page cache.
Swap Cache
Only modified (or dirty) pages are saved in the swap file.
So long as these pages are not modified after they have been written to the swap file then the next time the page is swapped out there is no need to write it to the swap file as the page is already in the swap file. Instead the page can simply be discarded
Hardware Caches
One commonly implemented hardware cache is in the processor; a cache of Page Table Entries. In this case, the processor does not always read the page table directly but instead caches translations for pages as it needs them. These are the Translation Look-aside Buffers and contain cached copies of the page table entries from one or more processes in the system.
Memory Mapping
When an image is executed, the contents of the executable image must be brought into the processes virtual address space. The same is also true of any shared libraries that the executable image has been linked to use. The executable file is not actually brought into physical memory, instead it is merely linked into the processes virtual memory. Then, as the parts of the program are referenced by the running application, the image is brought into memory from the executable image. This linking of an image into a processes virtual address space is known as memory mapping.

Areas of Virtual Memory
Every processes virtual memory is represented by an mm_struct data structure. This contains information about the image that it is currently executing (for example bash) and also has pointers to a number of vm_area_struct data structures. Each vm_area_struct data structure describes the start and end of the area of virtual memory, the processes access rights to that memory and a set of operations for that memory. These operations are a set of routines that Linux must use when manipulating this area of virtual memory. For example, one of the virtual memory operations performs the correct actions when the process has attempted to access this virtual memory but finds (via a page fault) that the memory is not actually in physical memory. This operation is the nopage operation. The nopage operation is used when Linux demand pages the pages of an executable image into memory.

Demand Paging
Once an executable image has been memory mapped into a processes virtual memory it can start to execute. As only the very start of the image is physically pulled into memory it will soon access an area of virtual memory that is not yet in physical memory. When a process accesses a virtual address that does not have a valid page table entry, the processor will report a page fault to Linux.
The page fault describes the virtual address where the page fault occurred and the type of memory access that caused.

Dynamic memory allocation
In computer science, dynamic memory allocation (also known as heap-based memory allocation) is the allocation of memory storage for use in a computer program during the runtime of that program. It can be seen also as a way of distributing ownership of limited memory resources among many pieces of data and code.
Dynamically allocated memory exists until it is released either explicitly by the programmer, or by the garbage collector. This is in contrast to static memory allocation, which has a fixed duration. It is said that an object so allocated has a dynamic lifetime.
Garbage collection
In computer science, garbage collection (GC) is a form of automatic memory management. It is a special case of resource management, in which the limited resource being managed is memory. The garbage collector, or just collector, attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program. Garbage collection was invented by John McCarthy around 1959 to solve problems in Lisp.
Memory management unit
A memory management unit (MMU), sometimes called paged memory management unit (PMMU), is a computer hardware component responsible for handling accesses to memory requested by the CPU. Its functions include translation of virtual addresses to physical addresses (i.e., virtual memory management), memory protection, cache control, bus arbitration, and, in simpler computer architectures (especially 8-bit systems), bank switching.
Page table
A page table is the data structure used by a virtual memory system in a computer operating system to store the mapping between virtual addresses and physical addresses. Virtual addresses are those unique to the accessing process. Physical addresses are those unique to the CPU, i.e., RAM.
Paging
This article is about computer virtual memory. For the wireless communication devices, see pager. Bank switching is also sometimes referred to as paging. Page flipping is also sometimes referred to as paging.
In computer operating systems there are various ways in which the operating system can store and retrieve data from secondary storage for use in main memory. One such memory management scheme is referred to as paging. In the paging memory-management scheme, the operating system retrieves data from secondary storage in same-size blocks called pages. The main advantage of paging is that it allows the physical address space of a process to be noncontiguous.
Virtual memory

The program thinks it has a large range of contiguous addresses; but in reality the parts it is currently using are scattered around RAM, and the inactive parts are saved in a disk file.
Virtual memory is a computer system technique developed at the University of Manchester, which gives an application program the impression that it has contiguous working memory (an address space), while in fact it may be physically fragmented and may even overflow on to disk storage.
Developed for multitasking kernels, virtual memory provides two primary functions:
Each process has its own address space, thereby not required to be relocated nor required to use relative addressing mode.
Each process sees one contiguous block of free memory upon launch. Fragmentation is hidden.

Process Scheduling

PROCESS STATE DIAGRAM



PROCESS SCHEDULING

WHAT IS A PROCESS?

A process can simply be defined as a program in execution. it can be defined as a program currently making use of the processor at any one time. The diagram below shows the various states of a process:

A process can be on any of the following states:
Ready: This is when the process is ready to be run on the processor.
Running: This is when the process is currently making use of the processor.
Blocked: This is when the process is waiting for an input such as user response or data from another process. A process may be in the blocked state if it needs to access a resource.
Other variations of the above named states are:
Ready Suspend: This is when a process is swapped out of a memory by Memory Management system in order to free memory for other process.
Blocked Suspend: This is when a process is swapped out of memory after incurring an O/I wait
Terminate: This is when a process has finished its run.

Summary

• Only one process at a time is running on the CPU


• Process gives up CPU:


• If it starts waiting for an event


• Otherwise: other processes need fair access


• OS schedules which ready process to run next


• Time slice or quantum for each process


• Scheduling algorithms


• affect performance
  

SCHEDULING

Scheduling is a key concept in computer multitasking, multiprocessing operating system and real-time operating system designs. Scheduling refers to the way processes are assigned to run on the available CPUs, since there are typically many more processes running than there are available CPUs. This assignment is carried out by software known as a scheduler or dispatcher.

SCHEDULER

The scheduler is concerned mainly with:
CPU utilization - to keep the CPU as busy as possible.
Throughput - number of processes that complete their execution per time unit.
Turnaround - total time between submission of a process and its completion.
Waiting time - amount of time a process has been waiting in the ready queue. Response Time- amount of time it takes from when a request was submitted until the first response is produced.
Fairness - Equal CPU time to each thread.

Types of schedulers
Operating systems may feature up to 3 distinct types of schedulers: a long-term scheduler (also known as an admission scheduler or high-level scheduler), a mid-term or medium-term scheduler and a short-term scheduler (also known as a dispatcher). The names suggest the relative frequency with which these functions are performed.

1. Long-term Scheduler
The long-term, or admission, scheduler decides which jobs or processes are to be admitted to the ready queue; that is, when an attempt is made to execute a program, its admission to the set of currently executing processes is either authorized or delayed by the long-term scheduler. Thus, this scheduler dictates what processes are to run on a system, and the degree of concurrency to be supported at any one time - ie: whether a high or low amount of processes are to be executed concurrently, and how the split between IO intensive and CPU intensive processes is to be handled. In modern OS's, this is used to make sure that real time processes get enough CPU time to finish their tasks. Without proper real time scheduling, modern GUI interfaces would seem sluggish.

2. Mid-term Scheduler
The mid-term scheduler temporarily removes processes from main memory and places them on secondary memory (such as a disk drive) or vice versa. This is commonly referred to as "swapping out" or "swapping in" (also incorrectly as "paging out" or "paging in"). The mid-term scheduler may decide to swap out a process which has not been active for some time, or a process which has a low priority, or a process which is page faulting frequently, or a process which is taking up a large amount of memory in order to free up main memory for other processes, swapping the process back in later when more memory is available, or when the process has been unblocked and is no longer waiting for a resource.

In many systems today (those that support mapping virtual address space to secondary storage other than the swap file), the mid-term scheduler may actually perform the role of the long-term scheduler, by treating binaries as "swapped out processes" upon their execution. In this way, when a segment of the binary is required it can be swapped in on demand, or "lazy loaded".

3. Short-term Scheduler

The short-term scheduler (also known as the CPU scheduler) decides which of the ready, in-memory processes are to be executed (allocated a CPU) next following a clock interrupt, an IO interrupt, an operating system call or another form of signal. Thus the short-term scheduler makes scheduling decisions much more frequently than the long-term or mid-term schedulers - a scheduling decision will at a minimum have to be made after every time slice, and these are very short. This scheduler can be preemptive, implying that it is capable of forcibly removing processes from a CPU when it decides to allocate that CPU to another process, or non-preemptive (also known as "voluntary" or "co-operative"), in which case the scheduler is unable to "force" processes off the CPU.

Dispatcher
Another component involved in the CPU-scheduling function is the dispatcher. The dispatcher is the module that gives control of the CPU to the process selected by the short-term scheduler. This function involves the following:

• Switching context


• Switching to user mode
  

Jumping to the proper location in the user program to restart that program The dispatcher should be as fast as possible, since it is invoked during every process switch. The time it takes for the dispatcher to stop one process and start another running is known as the dispatch latency.

Scheduling criteria
Different CPU scheduling algorithms have different properties, and the choice of a particular algorithm may favor one class of processes over another. In choosing which algorithm to use in a particular situation, we must consider the properties of the various algorithms. Many criteria have been suggested for comparing CPU scheduling algorithms. Which characteristics are used for comparison can make a substantial difference in which algorithm is judged to be best. The criteria include the following:
1. CPU Utilization. We want to keep the CPU as busy as possible.
2. Throughput. If the CPU is busy executing processes, then work is being done. One measure of work is the number of processes that are completed per time unit, called throughput. For long processes, this rate may be one process per hour; for short transactions, it may be 10 processes per second.
3. Turnaround time. From the point of view of a particular process, the important criterion is how long it takes to execute that process. The interval from the time of submission of a process to the time of completion is the turnaround time. Turnaround time is the sum of the periods spent waiting to get into memory, waiting in the ready queue, executing on the CPU, and doing I/O.
4. Waiting time. The CPU scheduling algorithm does not affect the amount of the time during which a process executes or does I/O; it affects only the amount of time that a process spends waiting in the ready queue. Waiting time is the sum of periods spend waiting in the ready queue.
5. Response time. In an interactive system, turnaround time may not be the best criterion. Often, a process can produce some output fairly early and can continue computing new results while previous results are being output to the user. Thus, another measure is the time from the submission of a request until the first response is produced. This measure, called response time, is the time it takes to start responding, not the time it takes to output the response. The turnaround time is generally limited by the speed of the output device.
It is desirable to maximize CPU utilization and throughput and to minimize turnaround time, waiting time, and response time. In most cases, we optimize the average measure. However, under some circumstances, it is desirable to optimize the minimum or maximum values rather than the average. For example, to guarantee that all users get good service, we may want to minimize the maximum response time. Investigators have suggested that, for interactive systems, it is more important to minimize the variance in the response time than to minimize the average response time. A system with reasonable and predictable response time may be considered more desirable than a system that is faster on the average but is highly variable. However, little work has been done on CPU-scheduling algorithms that minimize variance.

CPU-bound
most of its time doing computation - little I/O
I/O-bound
most of its time doing I/O - little computation
Multilevel scheduling
Classified into different groups :

• foreground (interactive) vs.
  
• background (batch)
  

each group has its own ready queue

Preemptive Vs Nonpreemptive Scheduling
The Scheduling algorithms can be divided into two categories with respect to how they deal with clock interrupts.

• Nonpreemptive Scheduling
  A scheduling discipline is nonpreemptive if, once a process has been given the CPU, the CPU cannot be taken away from that process.
  Following are some characteristics of nonpreemptive scheduling
  In nonpreemptive system, short jobs are made to wait by longer jobs but the overall treatment of all processes is fair.
  In nonpreemptive system, response times are more predictable because incoming high priority jobs can not displace waiting jobs.
  In nonpreemptive scheduling, a schedular executes jobs in the following two situations.
  When a process switches from running state to the waiting state.
  When a process terminates.

• Preemptive Scheduling
  A scheduling discipline is preemptive if, once a process has been given the CPU can taken away.
  The strategy of allowing processes that are logically runable to be temporarily suspended is called Preemptive Scheduling and it is contrast to the "run to completion" method.
  

Presentation Data (HPFS & Ext3)

HPFS and Ext3

High Performance File System
HPFS or High Performance File System is a file system created specifically for the OS/2 operating system to improve upon the limitations of the FAT file system. It was written by Gordon Letwin and others at Microsoft and added to OS/2 version 1.2, at that time still a joint undertaking of Microsoft and IBM.

The HPFS file system was first introduced with OS/2 1.2 to allow for greater access to the larger hard drives that were then appearing on the market. Additionally, it was necessary for a new file system to extend the naming system, organization, and security for the growing demands of the network server market. HPFS maintains the directory organization of FAT, but adds automatic sorting of the directory based on filenames. Filenames are extended to up to 254 double byte characters. HPFS also allows a file to be composed of "data" and special attributes to allow for increased flexibility in terms of supporting other naming conventions and security. In addition, the unit of allocation is changed from clusters to physical sectors (512 bytes), which reduce lost disk space.

Under HPFS, directory entries hold more information than under FAT. As well as the attribute file, this includes information about the modification, creation, and access date and times. Instead of pointing to the first cluster of the file, the directory entries under HPFS point to the FNODE. The FNODE can contain the file's data, or pointers that may point to the file's data or to other structures that will eventually point to the file's data.

HPFS attempts to allocate as much of a file in contiguous sectors as possible. This is done in order to increase speed when doing sequential processing of a file.

HPFS organizes a drive into a series of 8 MB bands, and whenever possible a file is contained within one of these bands. Between each of these bands are 2K allocation bitmaps, which keep track of which sectors within a band have and have not been allocated. Banding increases performance because the drive head does not have to return to the logical top (typically cylinder 0) of the disk, but to the nearest band allocation bitmap to determine where a file is to be stored.

Additionally, HPFS includes a couple of unique special data objects:
1. Super Block
The Super Block is located in logical sector 16 and contains a pointer to the FNODE of the root directory. One of the biggest dangers of using HPFS is that if the Super Block is lost or corrupted due to a bad sector, so are the contents of the partition, even if the rest of the drive is fine. It would be possible to recover the data on the drive by copying everything to another drive with a good sector 16 and rebuilding the Super Block. However, this is a very complex task.

2. Spare Block
The Spare Block is located in logical sector 17 and contains a table of "hot fixes" and the Spare Directory Block. Under HPFS, when a bad sector is detected, the "hot fixes" entry is used to logically point to an existing good sector in place of the bad sector. This technique for handling write errors is known as hot fixing. Hot fixing is a technique where if an error occurs because of a bad sector, the file system moves the information to a different sector and marks the original sector as bad. This is all done transparent to any applications that are performing disk I/O (that is, the application never knows that there were any problems with the hard drive). Using a file system that supports hot fixing will eliminate error messages such as the FAT "Abort, Retry, or Fail?"error message that occurs when a bad sector is encountered.

Note: The version of HPFS that is included with Windows NT does not support hot fixing.

Among its improvements are:

• support for mixed case file names, in different code pages
• support for long file names (256 characters as opposed to FAT's 8+3 characters)
• more efficient use of disk space (files are not stored using multiple-sector clusters but on a per-sector basis)
• an internal architecture that keeps related items close to each other on the disk volume
• less fragmentation of data
• extent-based space allocation
• separate datestamps for last modification, last access, and creation (as opposed to FAT's one last modification datestamp)
• a B+ tree structure for directories
• root directory located at the mid-point, rather than beginning of the disk, for faster average access
• HPFS also can keep 64 KB of metadata ("extended attributes") per file.
  

IBM offers two kind of IFS drivers for this file system:

• the standard one with a cache limited to 2 MB
• HPFS386 provided with the server versions of OS/2

Windows Native Support
Windows 95 and its successors Windows 98, Windows Me can read/write HPFS only when mapped via a network share, but cannot read it from a local disk. They listed the NTFS partitions of networked computers as "HPFS", because NTFS and HPFS share the same filesystem identification number in the partition table.

Windows NT 3.1 and 3.5 have native read/write support for local disks and can even be installed onto an HPFS partition. This is because NT was originally going to be a version of OS/2.

Windows NT 3.51 can also read and write from local HPFS formatted drives. However, Microsoft discouraged using HPFS in Windows NT 4 and in subsequent versions. Microsoft even removed the ability of NT 3.51 to format an HPFS file system. Starting with Windows NT 4 the file system driver pinball.sys enabling the read/write access is not included in a default installation anymore. Pinball.sys is included on the installation media for Windows 2000 and can be manually installed and used with some limitations. Later Windows versions do not ship with this driver.

Microsoft retained rights to OS/2 technologies, including the HPFS file system, after they ceased collaboration. Since Windows NT 3.1 was designed for more rigorous (enterprise-class) use than previous versions of Windows, it included support for HPFS (and NTFS) giving it a larger storage capacity than FAT file systems. However, since HPFS lacks a journal, any recovery after an unexpected shutdown or other error state takes progressively longer as the file system grows. A utility such as CHKDSK would need to scan each entry in the file system to ensure no errors are present, a problem which is vastly reduced on NTFS where the journal is simply replayed.

Advantages of HPFS

• HPFS is best for drives in the 200-400 MB range.
• Support for long file names upto 256 characters.
• Upper and lower case- HPFS preserves case, but it is not case sensitive
• Native support for EA’s FAT is just too fragile to support this and the workplace spell depends on it heavily.
• HPFS provides high performance.
• Much greater integrity: Signature at the beginning of the system structure sectors, forwards and backwards links in fnode trees.
• Much less fragmentation.

Disadvantages of HPFS

• Because of the overhead involved in HPFS, it is not a very efficient choice for a volume of under approximately 200 MB. In addition, with volumes larger than about 400 MB, there will be some performance degradation.
• You cannot set security on HPFS under WindowsNT.
• HPFS is only supported under Windows NT versions 3.1, 3.5, and 3.51. Windows NT 4.0 cannot access HPFS partitions.

Ext3

The ext3 or third extended file system is a journaled file system that is commonly used by the Linux kernel. It is the default file system for many popular Linux distributions. Stephen Tweedie first revealed that he was working on extending ext2 in Journaling the Linux ext2fs File system in a 1998 paper and later in a February 1999 kernel mailing list posting, and the file system was merged with the mainline Linux kernel in November 2001 from 2.4.15 onward. Its main advantage over ext2 is journaling which improves reliability and eliminates the need to check the file system after an unclean shutdown. Its successor is ext4.

Journaling results in massively reduced time spent recovering a file system after a crash, and is therefore in high demand in environments where high availability is important, not only to improve recovery times on single machines but also to allow a crashed machine's file system to be recovered on another machine when we have a cluster of nodes with a shared disk.

Advantages
Although its performance (speed) is less attractive than competing Linux file systems such as JFS, ReiserFS and XFS, it has a significant advantage in that it allows in-place upgrades from the ext2 file system without having to back up and restore data. Ext3 also uses less CPU power than ReiserFS and XFS. It is also considered safer than the other Linux file systems due to its relative simplicity and wider testing base.

The ext3 file system adds, over its predecessor:

• A Journaling file system
• Online file system growth
• Htree indexing for larger directories. An HTree is a specialized version of a B-tree (not to be confused with the H tree fractal).
  

Without these, any ext3 file system is also a valid ext2 file system. This has allowed well-tested and mature file system maintenance utilities for maintaining and repairing ext2 file systems to also be used with ext3 without major changes. The ext2 and ext3 file systems share the same standard set of utilities, e2fsprogs, which includes a fsck tool. The close relationship also makes conversion between the two file systems (both forward to ext3 and backward to ext2) straightforward.

While in some contexts the lack of "modern" file system features such as dynamic inode allocation and extents could be considered a disadvantage, in terms of recoverability this gives ext3 a significant advantage over file systems with those features. The file system metadata is all in fixed, well-known locations, and there is some redundancy inherent in the data structures that may allow ext2 and ext3 to be recoverable in the face of significant data corruption, where tree-based file systems may not be recoverable.

What is a Journaling File system?

A journaling file system keeps a journal or log of the changes that are being made to the file system during disk writing that can be used to rapidly reconstruct corruptions that may occur due to events such a system crash or power outage. The level of journaling performed by the file system can be configured to provide a number of levels of logging depending on your needs and performance requirements.


What are the Advantages of a Journaling File system?

There are a number of advantages to using a journaling files system:

Both the size and volume of data stored on disk drives has grown exponentially over the years. The problem with a non-journaled file system is that following a crash the fsck (file system consistency check) utility has to be run. fsck will scan the entire file system validating all entries and making sure that blocks are allocated and referenced correctly. If it finds a corrupt entry it will attempt to fix the problem. The issues here are two-fold. Firstly, the fsck utility will not always be able to repair damage and you will end up with data in the lost+found directory. This is data that was being used by an application but the system no longer knows where they were reference from. The other problem is the issue of time. It can take a very long time to complete the fsck process on a large file system leading to unacceptable down time.

A journaled file system records information in a log area on a disk (the journal and log do not need to be on the same device) during each write. This is a essentially an "intent to commit" data to the file system. The amount of information logged is configurable and ranges from not logging anything, to logging what is known as the "metadata" (i.e ownership, date stamp information etc), to logging the "metadata" and the data blocks that are to be written to the file. Once the log is updated the system then writes the actual data to the appropriate areas of the file system and marks an entry in the log to say the data is committed.

After a crash the file system can very quickly be brought back on-line using the journal log reducing what could take minutes using fsck to seconds with the added advantage that there is considerably less chance of data loss or corruption.


What is a Journal Checkpoint?

When a file is accessed on the filesystem, the last snapshot of that file is read from the disk into memory. The journal log is then consulted to see if any uncommitted changes have been made to the file since the data was last written to the file (essentially looking for an "intention to commit" in the log entry as described above). At particular points the filesystem will update file data on the disk from the uncommited log entries and trim those entries from the log. Committing operations from the log and synchronizing the log and its associated filesystem is called a checkpoint.

What are the disadvantages of a Journaled Filesystem?

Nothing in life is is free and ext3 and journaled filesystems are no exception to the rule. The biggest draw back of journaling is in the area of performance simply because more disk writes are required to store information in the log. In practice, however, unless you are running system where disk performance is absolutely critical the performance difference will be negligable.

What Journaling Options are Available with the ext3 filesystem?

The ext3 file system provides three options. These are as follows:

Journal (lowest risk)

Both metadata and file contents are written to the journal before being committed to the main file system. Because the journal is relatively continuous on disk, this can improve performance in some circumstances. In other cases, performance gets worse because the data must be written twice - once to the journal, and once to the main part of the file system.

Ordered (medium risk)

Only metadata is journaled; file contents are not, but it's guaranteed that file contents are written to disk before associated metadata is marked as committed in the journal. This is the default on many Linux distributions. If there is a power outage or kernel panic while a file is being written or appended to, the journal will indicate the new file or appended data has not been "committed", so it will be purged by the cleanup process. (Thus appends and new files have the same level of integrity protection as the "journaled" level.) However, files being overwritten can be corrupted because the original version of the file is not stored. Thus it's possible to end up with a file in an intermediate state between new and old, without enough information to restore either one or the other (the new data never made it to disk completely, and the old data is not stored anywhere). Even worse, the intermediate state might intersperse old and new data, because the order of the write is left up to the disk's hardware. XFS uses this form of journaling.

Writeback (highest risk)

Only metadata is journaled; file contents are not. The contents might be written before or after the journal is updated. As a result, files modified right before a crash can become corrupted. For example, a file being appended to may be marked in the journal as being larger than it actually is, causing garbage at the end. Older versions of files could also appear unexpectedly after a journal recovery. The lack of synchronization between data and journal is faster in many cases. JFS uses this level of journaling, but ensures that any "garbage" due to unwritten data is zeroed out on reboot.

Does the Journal log have to be on the same disk as the file system?

No, the ext3 journal log does not have to be on the same physical device as the file system it is logging. On a Red Hat Linux the journal device can be specified using the journal_device= option with the -journal-options command line argument of the tune2fs utility.

Features of ext3

The ext3 file system is essentially an enhanced version of the ext2 file system. These improvements provide the following advantages:

Availability

After an unexpected power failure or system crash (also called an unclean system shutdown), each mounted ext2 file system on the machine must be checked for consistency by the e2fsck program. This is a time-consuming process that can delay system boot time significantly, especially with large volumes containing a large number of files. During this time, any data on the volumes is unreachable.

The journaling provided by the ext3 file system means that this sort of file system check is no longer necessary after an unclean system shutdown. The only time a consistency check occurs using ext3 is in certain rare hardware failure cases, such as hard drive failures. The time to recover an ext3 file system after an unclean system shutdown does not depend on the size of the file system or the number of files; rather, it depends on the size of the journal used to maintain consistency. The default journal size takes about a second to recover, depending on the speed of the hardware.

Data Integrity

The ext3 file system provides stronger data integrity in the event that an unclean system shutdown occurs. The ext3 file system allows you to choose the type and level of protection that your data receives. By default, Red Hat Linux 8.0 configures ext3 volumes to keep a high level of data consistency with regard to the state of the file system.

Speed

Despite writing some data more than once, ext3 has a higher throughput in most cases than ext2 because ext3's journaling optimizes hard drive head motion. You can choose from three journaling modes to optimize speed, but doing so means trade offs in regards to data integrity.

Easy Transition

It is easy to change from ext2 to ext3 and gain the benefits of a robust journaling file system without reformatting.

Why ext3?
Ext3 is forward and backward compatible with ext2, allowing users to keep existing file systems while very simply adding journaling capability. Any user who wishes to un-journal a file system can do so easily (not that we expect many to do so...). Furthermore, an ext3 file system can be mounted as ext2 without even removing the journal, as long as a recent version of e2fsprogs (such as the one included in Red Hat Linux 7.2) is installed.
Ext3 benefits from the long history of fixes and enhancements to the ext2 file system, and will continue to do so. This means that ext3 shares ext2's well-known robustness, but also that as new features are added to ext2, they can be carried over to ext3 with little difficulty. When, for example, extended attributes or HTrees are added to ext2, it will be relatively easy to add them to ext3. (The extended attributes feature will enable things like access control lists; HTrees make directory operations extremely fast and highly scalable to very large directories.)
Ext3, like ext2, has a multi-vendor team of developers who develop it and understand it well; its development does not depend on any one person or organization.
Ext3 provides and makes use of a generic journaling layer (jbd) which can be used in other contexts. ext3 can journal not only within the file system, but also to other devices, so as NVRAM devices become available and supported under Linux, ext3 will be able to support them.
Ext3 has multiple journaling modes. It can journal all file data and metadata (data=journal), or it can journal metadata but not file data (data=ordered or data=writeback). When not journaling file data, you can choose to write file system data before metadata (data=ordered; causes all metadata to point to valid data), or not to handle file data specially at all (data=writeback; file system will be consistent, but old data may appear in files after an unclean system shutdown). This gives the administrator the power to make the tradeoff between speed and file data consistency, and to tune speed for specialized usage patterns.
Ext3 has broad cross-platform compatibility, working on 32- and 64- bit architectures, and on both little-endian and big-endian systems. Any system (currently including many Unix clones and variants, BeOS, and Windows) capable of accessing files on an ext2 file system will also be able to access files on an ext3 file system.
Ext3 does not require extensive core kernel changes and requires no new system calls, thus presenting Linus Torvalds no challenges that would effecitvely prevent him from integrating ext3 into his official Linux kernel releases. Ext3 is already integrated into Alan Cox's -ac kernels, slated for migration to Linus's official kernel soon.
The e2fsck file system recovery program has a long and proven track record of successful data recovery when software or hardware faults corrupt a file system. ext3 uses this same e2fsck code for salvaging the file system after such corruption, and therefore it has the same robustness against catastrophic data loss as ext2 in the presence of data-corruption faults.

Size limits
Ext3 has a maximum size for both individual files and the entire filesystem. For the filesystem as a whole that limit is 232 blocks. Both limits are dependent on the block size of the filesystem; the following chart summarizes the limits:

Block size
Max file size
Max filesystem size





1 KB
16 GB
2 TB

2 KB
256 GB
8 TB

4 KB
2 TB
16 TB

8 KB
2 TB
32 TB



Disadvantages
Functionality
Since ext3 aims to be backwards compatible with the earlier ext2, many of the on-disk structures are similar to those of ext2. Because of that, ext3 lacks a number of features of more recent designs, such as extents, dynamic allocation of inodes, and block sub allocation. There is a limit of 31998 sub-directories per one directory, stemming from its limit of 32000 links per inode.

ext3, like most current Linux filesystems, cannot be fsck-ed while the filesystem is mounted for writing. Attempting to check a file system that is already mounted may detect bogus errors where changed data has not reached the disk yet, and corrupt the file system in an attempt to "fix" these errors.

Defragmentation
There is no online ext3 defragmentation tool that works on the filesystem level. An offline ext2 defragmenter, e2defrag, exists but requires that the ext3 filesystem be converted back to ext2 first. But depending on the feature bits turned on in the filesystem, e2defrag may destroy data; it does not know how to treat many of the newer ext3 features.

There are userspace defragmentation tools like Shake and defrag. Shake works by allocating space for the whole file as one operation, which will generally cause the allocator to find contiguous disk space. It also tries to write files used at the same time next to each other. Defrag works by copying each file over itself. However they only work if the filesystem is reasonably empty. A true defragmentation tool does not exist for ext3.

That being said, as the Linux System Administrator Guide states, "Modern Linux filesystem(s) keep fragmentation at a minimum by keeping all blocks in a file close together, even if they can't be stored in consecutive sectors. Some filesystems, like ext3, effectively allocate the free block that is nearest to other blocks in a file. Therefore it is not necessary to worry about fragmentation in a Linux system."

While ext3 is more resistant to file fragmentation than the FAT filesystem, nonetheless ext3 filesystems can get fragmented over time or on specific usage patterns, like slowly-writing large files. Consequently the successor to the ext3 filesystem, ext4, includes a filesystem defragmentation utility and support for extents (contiguous file regions).

Recovery
There is no support of deleted file recovery in file system design. Ext3 driver actively deletes files by wiping file inodes for crash safety reasons. That's why accidental 'rm -rf ...' may cause permanent data loss.

There are still several techniques and some commercial software like UFS Explorer Standard Recovery version 4 for recovery of deleted or lost files using file system journal analysis; however, they do not guarantee any specific file recovery.

There is no chance of file recovery after file system format.

Compression
Support for transparent compression is available as an unofficial patch for ext3. This patch is a direct port of e2compr and still needs further development, it compiles and boots well with upstream kernels but journaling is not implemented yet. The current patch is named e3compr.

No checksumming in journal
Ext3 does not do checksumming when writing to the journal. If barrier=1 is not enabled as a mount option (in /etc/fstab), and if the hardware is doing out-of-order write caching, one runs the risk of severe filesystem corruption during a crash.

Consider the following scenario: If hard disk writes are done out-of-order (due to modern hard disks caching writes in order to amortize write speeds), it is likely that one will write a commit block of a transaction before the other relevant blocks are written. If a power failure or unrecoverable crash should occur before the other blocks get written, the system will have to be rebooted. Upon reboot, the file system will replay the log as normal, and replay the "winners" (transactions with a commit block, including the invalid transaction above which happened to be tagged with a valid commit block). The unfinished disk write above will thus proceed, but using corrupt journal data. The file system will thus mistakenly overwrite normal data with corrupt data while replaying the journal. There is a test program available to trigger the problematic behavior. If checksums had been used, where the blocks of the "fake winner" transaction were tagged with a mutual checksum, the file system could have known better and not replayed the corrupt data onto the disk. Journal checksumming has been added to EXT4.

EXT3 distribution

The EXT3 filesystem patch distributions and design papers are available from ftp://ftp.kernel.org/pub/linux/kernel/people/sct/ext3

Alternately, these materials are available from ftp://ftp.uk.linux.org/pub/linux/sct/fs/jfs/

The EXT3 author and maintainer, Stephen Tweedie, may be reached at sct@redhat.com