E4rat: Difference between revisions

From ArchWiki
(→‎/var/lib/e4rat/startup.log is not accessible: e4rat-preload annoying messages)
m (unhide section link)
 
(83 intermediate revisions by 35 users not shown)
Line 1: Line 1:
[[Category:Boot process]]
[[Category:Boot process]]
[[es:E4rat]]
[[de:e4rat]]
[[ja:E4rat]]
[[ru:E4rat]]
[[ru:E4rat]]
[[zh-CN:E4rat]]
[[zh-hans:E4rat]]
{{Article summary start}}
{{Related articles start}}
{{Article summary text|How to drastically reduce boot and log-into-X time for ext4 file systems using [http://e4rat.sourceforge.net/ the e4rat range of tools].}}
{{Related|Improve boot performance}}
{{Article summary heading|Related}}
{{Related|Preload}}
{{Article summary wiki|Improve Boot Performance}}
{{Related|Ext4}}
{{Article summary wiki|Preload}}
{{Related articles end}}
{{Article summary wiki|Readahead}}
{{Article summary wiki|Ureadahead}}
{{Article summary wiki|Ext4}}
{{Article summary heading|Forum threads}}
{{Article summary link|Main discussion|https://bbs.archlinux.org/viewtopic.php?id=115976}}
{{Article summary link|Improved e4rat-preload|https://bbs.archlinux.org/viewtopic.php?id=117776}}
{{Article summary end}}


e4rat stands for e4 'reduced access time' (ext4 file system only) and is a project by Andreas Rid and Gundolf Kiefer. The [http://e4rat.sourceforge.net/ e4rat range of tools] are comprised of e4rat-collect, e4rat-realloc and e4rat-preload.
[http://e4rat.sourceforge.net/ e4rat] stands for e4 'reduced access time' (ext4 file system only) and is a project by Andreas Rid and Gundolf Kiefer. The [http://e4rat.sourceforge.net/ e4rat range of tools] are comprised of e4rat-collect, e4rat-realloc and e4rat-preload.


Current version is 0.2.1
There is an alternative implementation called e4rat-lite, which appears to be more active and more optimised than the original e4rat project, see [[#e4rat-lite]].


==Process==
== Process ==
If you look at a classical [[bootchart]] you will notice that neither disk nor CPU are utilized fully during the boot process. e4rat changes this to make full use of both disk and CPU during boot process and thus reduce boot time drastically. It consists of three stages:
* '''e4rat-collect''' - collect files for a specified time (default 120 seconds but this can be adjusted)
* '''e4rat-realloc''' - reallocate files
* '''e4rat-preload''' - preload them


===Who benefits, who does not===
If you look at a classical [[bootchart]] you will notice that neither disk nor CPU are utilized fully during the boot process. e4rat changes this to make full use of both disk and CPU during boot process and thus reduce boot time drastically. Its usage consists of three stages:
* '''e4rat-collect''' - assemble a list of all files loaded during a specified time after boot (default 120 seconds but this can be adjusted)
* '''e4rat-realloc''' - reallocate all the files in the list to one continuous space on the hard disk
* '''e4rat-preload''' - preload them all (using multiple CPU cores) for quick booting and loading of applications


e4rat has proven to be extremely effective for typical single user set-ups which boot straight into X, perhaps even with a number of programs open. If you have a server set-up and boot only into the CLI your boot time decrease may not be as drastic. Users of SSD drives do not benefit because there are no moving parts and thus (almost) no disk latency - [[Ureadahead]] might be worth looking at.
=== Who benefits, who does not ===


{{Note|'''(to ureadahead users)''' The [http://e4rat.sourceforge.net/wiki/index.php/Main_Page#Ubuntu_and_ureadahead official e4rat manual] states that ureadahead conflicts with e4rat. This may be true for Ubuntu but using e4rat in conjunction with ureadahead does work on Arch Linux, although it does not speed up the boot process any further.}}
e4rat has proven to be extremely effective for typical single user set-ups which boot straight into X, perhaps even with a number of programs open. If you have a server set-up and boot only into the CLI your boot time decrease may not be as drastic. Users of SSD drives do not benefit because there are no moving parts and thus (almost) no disk latency.


It is always better to be safe than sorry. Just make backup if you cannot afford to lose data on your partition.
It is always better to be safe than sorry. Just make backup if you cannot afford to lose data on your partition.


==Installation==
== Installation ==
[[pacman|Install]] {{Pkg|e4rat}} from the [[Official Repositories]].
 
[[Install]] {{AUR|e4rat}}.
 
{{Note|
* In order to build it, you must first rebuild {{Pkg|audit}} from the [[ABS]] with {{ic|staticlibs}} option explicitly enabled. Simply installing the default {{Pkg|audit}} package will result in a build error.
* Audit needs these options to be enabled in the kernel configuration (CONFIG_AUDIT) together with support for auditing system calls (CONFIG_AUDITSYSCALL) also see [[Kernel/Arch build system]]. Probably you will need {{ic|1=audit=1}} to add to your kernel parameters.
}}
 
{{Tip|The [[#e4rat-lite]] does not need the rebuilded {{Pkg|audit}}, it works fine with the version from the official repos.
}}
 
== Getting it to work ==


==Getting it to work==
Now for the nitty-gritty:
Now for the nitty-gritty:
===e4rat-collect===
To have e4rat collect a list of files you will need to append the following to your linux (or 'kernel') line in your '''/boot/grub/menu.lst''' (grub legacy) or '''/boot/grub/grub.cfg''' (grub2) file. For example:


{{bc|1=kernel /vmlinuz0linux root=/dev/disk/by-label/ARCH init=/sbin/e4rat-collect ro 5}}
=== e4rat-collect ===
 
To have e4rat collect a list of files you will need to append {{ic|1=init=/sbin/e4rat-collect}} to your  [[kernel parameters]]. For example:
 
kernel /vmlinuz-linux root=/dev/disk/by-label/ARCH init=/sbin/e4rat-collect ro 5


This will only have to be done once so you may prefer to append this command on the grub command line itself.
This will only have to be done once so you may prefer to append this command on the grub command line itself.


Upon booting e4rat-collect will watch your system for a default of 120 seconds. So if you boot, log into X, open your favourite browser and email client all within 2 minutes, every one of those activities is logged. To change the default of 120 seconds edit '''/etc/e4rat.conf'''. To manually stop e4rat-collect type:
Upon booting e4rat-collect will watch your system for a default of 120 seconds. So if you boot, log into X, open your favourite browser and email client all within 2 minutes, every one of those activities is logged. To change the default of 120 seconds edit {{ic|/etc/e4rat.conf}}. To manually stop e4rat-collect type:
 
e4rat-collect -k
 
or
 
pkill e4rat-collect


{{bc|e4rat-collect -k}}or{{bc|pkill e4rat-collect}}
Upon successful boot and after having waited the allotted time you should see the file {{ic|/var/lib/e4rat/startup.log}}.


Upon successful boot and after having waited the allotted time you should see the following file: '''/var/lib/e4rat/startup.log'''
Do not forget to remove the e4rat-collect command from your [[Boot loader]] configuration file (not necessary if you inserted it on the grub command line).


Do not forget to remove the e4rat-collect command from your '''menu.lst''' or '''grub.cfg''' file (not necessary if you inserted it on the grub command line).
=== e4rat-realloc ===


===e4rat-realloc===
Once ''e4rat-collect'' has finished to run, do:
For the reallocation process change to init 1
# e4rat-realloc /var/lib/e4rat/startup.log
{{bc|sudo init 1}}
Log in as root and run:
{{bc|e4rat-realloc /var/lib/e4rat/startup.log}}
This can take a while depending on how many files you have in your startup.log file.
This can take a while depending on how many files you have in your startup.log file.
===e4rat-preload===
Switching to rescue mode with {{ic|systemctl isolate rescue}} ([[Systemd#Mapping between SysV runlevels and systemd targets]]) may allow for more inodes/blocks to be reallocated as some may not be free while in ''multiuser.target''
Append the following permanently to your kernel line in your '''/boot/grub/menu.lst''' (grub legacy) or '''/boot/grub/grub.cfg''' (grub2) file:
 
{{Note|
* It may be worthwhile to repeat the reallocation step multiple times before exiting or rebooting in order to further reduce the fragmentation count. Simply re-run the command  a few times to see if this is possible on your setup. If so you will see the count number reduced after a few runs. This is perfectly safe and should not cause any issues with booting.
* Users using SysV-style [[init]] systems must switch to runlevel 1 using {{ic|sudo init 1}} prior to running {{ic|e4rat-realloc}}}}


{{bc|1=init=/sbin/e4rat-preload}}
=== e4rat-preload ===


{{Note|1=If you are using grub2, kernel parameters are added in {{Ic|/etc/default/grub}} - {{Ic|1=GRUB_CMDLINE_LINUX="..."}}}}
Append {{ic|1=init=/sbin/e4rat-preload}} permanently to your [[kernel parameters]].


===Alternative: e4rat-preload-lite===
== e4rat and init systems ==
An alternative preload binary has been developed by [https://bbs.archlinux.org/viewtopic.php?id=117776&p=1 jlindgren], it saves a few extra seconds from your boot time.


The savings come from
e4rat-collect defaults to replacing itself with {{ic|/sbin/init}} upon completion. With the default {{ic|systemd}} installation, this file is a symbolic link to {{ic|/lib/systemd/systemd}}. If you need to specify another process with PID 1, such as {{ic|/usr/bin/busybox}}, you can change this in {{ic|/etc/e4rat.conf}} by setting the {{ic|init}} parameter:
* using pure C with no external library dependencies, which drops the number of linked .so files from 22 to 3
init /usr/bin/busybox
Note: Current [0.2.2] version of e4rat-preload is linked against 5 .so libraries, including libc, libm, libpthread ! So there is not much of a difference here !
* preloading only the first 100 files (both inodes and file contents) before starting /sbin/init, then continuing to load the remaining files in parallel with the normal boot sequence.


You can install {{AUR|e4rat-preload-lite}} from the [[AUR]].
This allows to launch both {{ic|e4rat-preload}} and {{ic|bootchart}} in the same boot sequence.


Append (or replace) the following permanently to your kernel line in your '''/boot/grub/menu.lst''' (grub legacy) or '''/boot/grub/grub.cfg''' (grub2) file:
== e4rat-lite ==


{{bc|1=init=/usr/sbin/e4rat-preload-lite}}
An alternative to e4rat with some improvements made. It is also expected to circumvent some issues one may experience with the original e4rat package.
It can be acquired from {{AUR|e4rat-lite-git}}.


Reboot and enjoy.
=== Using e4rat-lite ===


==e4rat with different init system==
The commands for e4rat-lite work almost identically to e4rat. Look at [https://github.com/ShyPixie/e4rat-lite#how-to-use the README of the project]. The summary:
e4rat-collect defaults to replacing itself with /sbin/init upon completion. If you need to specify another PID 1, such as /bin/systemd, you can change this in /etc/e4rat.conf by setting the '''init''' parameter and uncommenting the line.


==Bootchart==
collect: {{ic| 1=init=/usr/bin/e4rat-lite-collect}} - also default limit 120 seconds, you can also end the collection session using:


<b>Note: this has not worked for and is still in development - any suggestions welcome</b>
# e4rat-lite-collect -k


You will see a noticeable improvement but nothing can beat a nice [[Bootchart]]. Have it run before and after e4rat installation and gawk at the difference.
realloc: {{ic| /usr/bin/e4rat-lite-realloc}} - the switch to runlevel 1 is recommended, if you use [[systemd]] run this beforehand:


===bootchart 0.9-9===
# systemctl isolate rescue.target


This version automatically stops logging as soon as [[Display Manager]] comes up. Supposedly the following overrides that and continues logging but it does not work for me:
preload: {{ic| 1=init=/usr/bin/e4rat-lite-preload}}


To continue logging adjust your '''/etc/bootchartd.conf''' as follows:
startup.log: {{ic| /var/lib/e4rat-lite/startup.log}}


{{bc|1=AUTO_STOP_LOGGER="no"}}
configuration file: {{ic| /etc/e4rat-lite.conf}} - if you use [[systemd]] change init_file property to:
 
init_file=/usr/lib/systemd/systemd
 
== Bootchart ==
 
[[Bootchart]] can be used to generate graphs of the system startup. This is useful to get visual representations of the CPU and Disk usages.
While not required, a before and after comparison of the boot process can be obtained using Bootchart.
 
=== bootchart ===
 
{{Note|The official bootchart package appears to be deprecated, as it no longer includes the bootchart-render command needed to generate the graph. It is recommended to use bootchart2 as an alternative.}}
 
This version of [[bootchart]] automatically stops logging as soon as a [[display manager]] comes up. The following overrides that and continues logging:
 
{{Accuracy|It has been reported that the following line does not work.}}
 
To continue logging adjust your {{ic|/etc/bootchartd.conf}} as follows:
 
AUTO_STOP_LOGGER="no"


To stop it manually type:
To stop it manually type:


{{bc|~# bootchartd stop}}
# bootchartd stop


To run both e4rat-preload and bootchart append the following to your grub kernel line:
To run both e4rat-preload and bootchart append the following to your grub kernel line:


{{bc|1=init=/sbin/bootchartd bootchart_init=/sbin/e4rat-preload}}
init=/sbin/bootchartd bootchart_init=/sbin/e4rat-preload


===bootchart2===
=== bootchart2 ===


To get bootchart2 working together with e4rat edit '''/sbin/bootchartd''' and replace the line init="/sbin/init" with "init="/sbin/e4rat-preload".
To get {{AUR|bootchart2}} working together with e4rat edit {{ic|/sbin/bootchartd2}} and replace the line where it says
init="/sbin/init"
with
init="/sbin/e4rat-preload"
This will allow you to measure your boot time with the information that Bootchart2 provides.


'''/etc/bootchartd.conf''' of bootchart2 is set up differently and the line
It is easy to set up when to stop bootchart2 (contrary to bootchart) by editing its configuration file {{ic|/etc/bootchartd2.conf}}. The line


  EXIT_PROC="kdm_greet xterm konsole gnome-terminal metacity mutter compiz ldm icewm-session enlightenment"
  EXIT_PROC="kdm_greet xterm konsole gnome-terminal metacity mutter compiz ldm icewm-session enlightenment"


can be adjusted or left empty for logging to be stopped manually rather than at a predetermined programme start.
can be configured to stop Bootchart2 logging when the specified application launches. Alternatively it may be left empty for the logging to be stopped manually.


==Troubleshooting==
To generate the chart, run the command: {{ic|pybootchartgui -i}}
 
== Troubleshooting ==


If things do not work you may want to try the following.
If things do not work you may want to try the following.


===startup.log is not created===
=== startup.log is not created ===
 
* Disable auditd service
* Check the following for any hints
# dmesg | grep e4rat
* Try to increase verbosity and loglevel to 31 in your {{ic|e4rat.conf}}
* Try using e4rat-lite instead of e4rat
 
=== e4rat erroneously reports an ext2 files system ===
 
Add {{ic|1=rootfstype=ext4}} to [[kernel parameters]] from your bootloader.


*comment out auditd from your '''rc.conf'''
=== /var/lib/e4rat/startup.log is not accessible ===
*check the following for any hints
dmesg | grep e4rat
*try to increase verbosity and loglevel to 31 in your '''e4rat.conf'''


===e4rat erroneously reports an ext2 files system===
This suggests that you have {{ic|/var}} on a separate partition which is not yet mounted during boot. You need move your {{ic|startup.log}} to an accessible partition ({{ic|/etc/e4rat/}} is just fine) and adjust your {{ic|/etc/e4rat.conf}} to reflect this change:
startup_log_file /etc/e4rat/startup.log


*add the following to the kernel line in your '''grub.cfg''' or '''menu.lst'''
=== Remove annoying message that mess up boot message ===
rootfstype=ext4


===/var/lib/e4rat/startup.log is not accessible===
If you are annoyed by the e4rat-preload message during boot, decrease verbose to 1 in {{ic|/etc/e4rat.conf}}


*this suggests that you have /var on a separate partition which is not yet mounted during boot. You need move your '''startup.log''' to an accessible partition (/etc/e4rat/ is just fine) and adjust your '''/etc/e4rat.conf''' to reflect this change:
== See also ==
{{bc|startup_log_file /etc/e4rat/startup.log}}


===Remove annoying message that mess up boot message===
* [https://bbs.archlinux.org/viewtopic.php?id=115976 Main discussion on the forum]
If you are annoyed by the e4rat-preload message during boot, decrease loglevel to 1 in /etc/e4rat.conf
* [https://bbs.archlinux.org/viewtopic.php?id=117776 Improved e4rat-preload - forum thread]

Latest revision as of 11:40, 4 January 2024

e4rat stands for e4 'reduced access time' (ext4 file system only) and is a project by Andreas Rid and Gundolf Kiefer. The e4rat range of tools are comprised of e4rat-collect, e4rat-realloc and e4rat-preload.

There is an alternative implementation called e4rat-lite, which appears to be more active and more optimised than the original e4rat project, see #e4rat-lite.

Process

If you look at a classical bootchart you will notice that neither disk nor CPU are utilized fully during the boot process. e4rat changes this to make full use of both disk and CPU during boot process and thus reduce boot time drastically. Its usage consists of three stages:

  • e4rat-collect - assemble a list of all files loaded during a specified time after boot (default 120 seconds but this can be adjusted)
  • e4rat-realloc - reallocate all the files in the list to one continuous space on the hard disk
  • e4rat-preload - preload them all (using multiple CPU cores) for quick booting and loading of applications

Who benefits, who does not

e4rat has proven to be extremely effective for typical single user set-ups which boot straight into X, perhaps even with a number of programs open. If you have a server set-up and boot only into the CLI your boot time decrease may not be as drastic. Users of SSD drives do not benefit because there are no moving parts and thus (almost) no disk latency.

It is always better to be safe than sorry. Just make backup if you cannot afford to lose data on your partition.

Installation

Install e4ratAUR.

Note:
  • In order to build it, you must first rebuild audit from the ABS with staticlibs option explicitly enabled. Simply installing the default audit package will result in a build error.
  • Audit needs these options to be enabled in the kernel configuration (CONFIG_AUDIT) together with support for auditing system calls (CONFIG_AUDITSYSCALL) also see Kernel/Arch build system. Probably you will need audit=1 to add to your kernel parameters.
Tip: The #e4rat-lite does not need the rebuilded audit, it works fine with the version from the official repos.

Getting it to work

Now for the nitty-gritty:

e4rat-collect

To have e4rat collect a list of files you will need to append init=/sbin/e4rat-collect to your kernel parameters. For example:

kernel /vmlinuz-linux root=/dev/disk/by-label/ARCH init=/sbin/e4rat-collect ro 5

This will only have to be done once so you may prefer to append this command on the grub command line itself.

Upon booting e4rat-collect will watch your system for a default of 120 seconds. So if you boot, log into X, open your favourite browser and email client all within 2 minutes, every one of those activities is logged. To change the default of 120 seconds edit /etc/e4rat.conf. To manually stop e4rat-collect type:

e4rat-collect -k

or

pkill e4rat-collect

Upon successful boot and after having waited the allotted time you should see the file /var/lib/e4rat/startup.log.

Do not forget to remove the e4rat-collect command from your Boot loader configuration file (not necessary if you inserted it on the grub command line).

e4rat-realloc

Once e4rat-collect has finished to run, do:

# e4rat-realloc /var/lib/e4rat/startup.log

This can take a while depending on how many files you have in your startup.log file. Switching to rescue mode with systemctl isolate rescue (Systemd#Mapping between SysV runlevels and systemd targets) may allow for more inodes/blocks to be reallocated as some may not be free while in multiuser.target

Note:
  • It may be worthwhile to repeat the reallocation step multiple times before exiting or rebooting in order to further reduce the fragmentation count. Simply re-run the command a few times to see if this is possible on your setup. If so you will see the count number reduced after a few runs. This is perfectly safe and should not cause any issues with booting.
  • Users using SysV-style init systems must switch to runlevel 1 using sudo init 1 prior to running e4rat-realloc

e4rat-preload

Append init=/sbin/e4rat-preload permanently to your kernel parameters.

e4rat and init systems

e4rat-collect defaults to replacing itself with /sbin/init upon completion. With the default systemd installation, this file is a symbolic link to /lib/systemd/systemd. If you need to specify another process with PID 1, such as /usr/bin/busybox, you can change this in /etc/e4rat.conf by setting the init parameter:

init /usr/bin/busybox

This allows to launch both e4rat-preload and bootchart in the same boot sequence.

e4rat-lite

An alternative to e4rat with some improvements made. It is also expected to circumvent some issues one may experience with the original e4rat package. It can be acquired from e4rat-lite-gitAUR.

Using e4rat-lite

The commands for e4rat-lite work almost identically to e4rat. Look at the README of the project. The summary:

collect: init=/usr/bin/e4rat-lite-collect - also default limit 120 seconds, you can also end the collection session using:

# e4rat-lite-collect -k

realloc: /usr/bin/e4rat-lite-realloc - the switch to runlevel 1 is recommended, if you use systemd run this beforehand:

# systemctl isolate rescue.target

preload: init=/usr/bin/e4rat-lite-preload

startup.log: /var/lib/e4rat-lite/startup.log

configuration file: /etc/e4rat-lite.conf - if you use systemd change init_file property to:

init_file=/usr/lib/systemd/systemd

Bootchart

Bootchart can be used to generate graphs of the system startup. This is useful to get visual representations of the CPU and Disk usages. While not required, a before and after comparison of the boot process can be obtained using Bootchart.

bootchart

Note: The official bootchart package appears to be deprecated, as it no longer includes the bootchart-render command needed to generate the graph. It is recommended to use bootchart2 as an alternative.

This version of bootchart automatically stops logging as soon as a display manager comes up. The following overrides that and continues logging:

The factual accuracy of this article or section is disputed.

Reason: It has been reported that the following line does not work. (Discuss in Talk:E4rat)

To continue logging adjust your /etc/bootchartd.conf as follows:

AUTO_STOP_LOGGER="no"

To stop it manually type:

# bootchartd stop

To run both e4rat-preload and bootchart append the following to your grub kernel line:

init=/sbin/bootchartd bootchart_init=/sbin/e4rat-preload

bootchart2

To get bootchart2AUR working together with e4rat edit /sbin/bootchartd2 and replace the line where it says

init="/sbin/init"

with

init="/sbin/e4rat-preload"

This will allow you to measure your boot time with the information that Bootchart2 provides.

It is easy to set up when to stop bootchart2 (contrary to bootchart) by editing its configuration file /etc/bootchartd2.conf. The line

EXIT_PROC="kdm_greet xterm konsole gnome-terminal metacity mutter compiz ldm icewm-session enlightenment"

can be configured to stop Bootchart2 logging when the specified application launches. Alternatively it may be left empty for the logging to be stopped manually.

To generate the chart, run the command: pybootchartgui -i

Troubleshooting

If things do not work you may want to try the following.

startup.log is not created

  • Disable auditd service
  • Check the following for any hints
# dmesg | grep e4rat
  • Try to increase verbosity and loglevel to 31 in your e4rat.conf
  • Try using e4rat-lite instead of e4rat

e4rat erroneously reports an ext2 files system

Add rootfstype=ext4 to kernel parameters from your bootloader.

/var/lib/e4rat/startup.log is not accessible

This suggests that you have /var on a separate partition which is not yet mounted during boot. You need move your startup.log to an accessible partition (/etc/e4rat/ is just fine) and adjust your /etc/e4rat.conf to reflect this change:

startup_log_file /etc/e4rat/startup.log

Remove annoying message that mess up boot message

If you are annoyed by the e4rat-preload message during boot, decrease verbose to 1 in /etc/e4rat.conf

See also