Talk:USB flash installation medium

From ArchWiki
Jump to navigation Jump to search

Graphical dd methods for Linux and Windows

Hi from Fedoraland, Arch folks! I just finished updating the Fedora wiki page for doing USB writing - with some references from the rather good SUSE pages on the same topic - main page, Windows, OS X. I think Arch, Fedora and SUSE all prefer a 'dd' style writing method, so we can take cues from each other here. The Fedora and SUSE pages both now feature ways to do a dd-style write on Windows and OS X, and the Fedora page has a rather convenient method for doing a dd-style write from GNOME - perhaps Arch might like to pick these up too? Could be handy for your users. I tested the recommended Windows tools (SUSE's Studio writer, and rawrite32) on a Windows laptop, and they both clearly did a clean direct write of the image, with Fedora's complex disk label setup to ensure BIOS, UEFI and Mac boot all work kept intact. i.e., they don't screw crap up like unetbootin or that Ubuntu tool does.

I suppose we could even think about providing a unified set of instructions / tools somewhere - there's quite a bit of redundancy going on at the moment. Thanks! AdamW (talk) 19:06, 30 January 2014 (UTC)

Hi AdamW, thank you for your work and for trying to involve our community! For the moment I've referenced the Fedora and openSUSE articles from our See also section, however let's keep this discussion open to see if somebody wants to adapt the methods we're missing for Arch on this article. We may even add some Expansion templates where needed, as reminders and to encourage contributions. In alternative, since in general instead of duplicating external sources we prefer to just provide links to them if they're well maintained, we may just add more specific links to the Fedora article in the proper sections.
Of course it would be great if we managed to reduce the redundancy among our articles with a unified article "somewhere", but the problem would be how to make it easy for our respective wiki users to edit such article without forcing them to register on another wiki (which clearly would instead have the effect of discouraging contributions). Maybe we could keep part of our articles synchronized with a bot?
-- Kynikos (talk) 04:25, 1 February 2014 (UTC)

Syslinux note

-- Moved from Help:Style, as the discussion shifted back to this article. -- Alad (talk) 19:46, 22 September 2014 (UTC)

@Ady (reopening): first of all I want to make clear that I didn't write the "own[ing]" argument in an accusing way, it was only meant to explain my point of view, I should have added a smiley at the end, I'll do it now :) More important, you said we "derail[ed] from the initial goal", but actually I asked you some questions that were very relevant and specific to your edit, and you haven't answered, so I'll paste them here again: Where exactly is that note inaccurate? Because usually when you feel some deeper explanations are out of place in some article it's because that article's topic is quite different, so the details should belong somewhere else: is it something regarding syslinux? Because we have an entire article for it, maybe you could add what's needed there and simply link there from the note? -- Kynikos (talk) 03:49, 17 August 2014 (UTC)

@Kynikos, Since the discussion was moved, the focus changed from the more-technical matter to whether using hidden text was/is appropriate (in this particular case). Thus, I thought that perhaps answering those questions here would have been considered off-topic.
The referenced Note says "The above step installs Syslinux's ldlinux.sys to the VBR of the USB partition, sets the partition as "active/boot" in the MBR partition table and writes the MBR boot code to the 1st 440-byte boot code region of the USB.". As you can see, this Note is not part of a practical step (and therefore, a slight inaccuracy is likely not going to affect a typical reader following the instructions). Yet, there is a purpose for this Note.
Users kept asking (including to upstream Syslinux) whether they can skip part of the steps, and just copy over ldlinux.sys (for example, to either install or update SYSLINUX in their USB drives). Moreover, some users didn't even ask; they simply (over)wrote this file and they complained that SYSLINUX (and/or Arch) fails to boot. So the Note is there to prevent this type of behavior / confusion / repeated waste of time. This explains why deleting the referenced Note would not be wise.
About the minor inaccuracy... What the SYSLINUX installer does (when performing the specific steps exactly as described in the relevant section of the article) is to:
  • copy ldlinux.{sys,c32};
  • patch the VBR so it can point to the correct block in the device (so ldlinux.sys can be found);
  • set the partition as "active/boot" in the partition table;
  • write the MBR boot code to the 1st 440-byte boot code region of the USB.
I want to clarify that the above description is only relevant for the specific section we are referring to. The SYSLINUX/EXTLINUX installers perform different tasks according to different cases.
So, there is a minor (technical) background inaccuracy in the Note.
Considering the goal of this Note (as I already described it above) in the context of this particular section of this particular article, and considering that the practical steps provided in the article are not affected by this minor inaccuracy, I asked my self (at the time I performed other editions on the article) whether I should also edit the Note. Would I be actually helping a typical reader that is interested in following the steps? With the objective to balance between accuracy and clarity for the typical reader, I decided not to over-complicate the Note with excessive details that in practical terms only would make the section longer without improving the outcome.
Adding a link to the Syslinux article would not improve the practical experience / steps for the user in this case. If anyone wants to find more technical information, they don't need yet another link from that same article to the same Syslinux page. In this article, the typical reader is (and/or should better be) likely focused on the practical steps.
Thus, my "hint" was "This note is not %100 technically accurate, but it is "accurate-enough" so to be able to perform the task.". For the typical reader trying to follow the practical steps, the procedure is accurate and he will complete the task successfully, so adding an "Accuracy" box would be counterproductive.
Generally speaking, I think there are cases where adding 'comments' (aka hidden text) is adequate, but I am perfectly fine following the guidelines in Help:Style. Lahwaacz's edition formatted the "hint" as "Accuracy" box, going exactly in the opposite direction of what the hint itself was saying (or at least, against the original intention of the comment). This was not beneficial for users, so I deleted the box.
Slightly long to read (apologies), but I hope this clarifies the matter. Ady (talk) 10:45, 17 August 2014 (UTC)
I see, thanks for explaining, then why not make explicit the reason why the note is there (KISS), like:
Note:
  • Make sure to run the above command in full, which installs Syslinux's ldlinux.sys to the VBR of the USB partition, sets the partition as "active/boot" in the MBR partition table and writes the MBR boot code to the 1st 440-byte boot code region of the USB. Only copying the file, for example in an attempt to update an existing Syslinux installation, will result in an unbootable device.
  • ...
This way nobody will see the note as useless and delete it.
-- Kynikos (talk) 13:50, 19 August 2014 (UTC)

"... and install the MBR or GPT partition table to the USB device as described in the page mentioned."

I am somewhat clueless about what page this instruction in USB_flash_installation_media#Using_manual_formatting is referring to, can anyone clear that up, what is "the page mentioned"? There is no further mentioning of GPT in the article. Also, not clear why it should be necessary to create a partition table on a USB device, which obviously is partitioned already (else, I wouldn't be able to manipulate any files on it). --Johannes Rohr (talk) 08:40, 5 February 2015 (UTC)

The previous link leads to Syslinux#Manual_install, have you looked there? -- Lahwaacz (talk) 09:46, 5 February 2015 (UTC)
Also, not clear why it should be necessary to create a partition table on a USB device, which obviously is partitioned already
I think this has to do with UEFI vs. BIOS booting. In the case of the more modern UEFI, you have a special FAT32-formatted boot partition which is labeled EF00 (so there are 2 partitions on the disk). If you're just going to create a BIOS boot USB, then you only need a single partition. Pgoetz (talk) 09:44, 5 February 2015 (UTC)
The wording has of the section has changed. Closing. -- nl6720 (talk) 09:12, 15 April 2021 (UTC)

Repeated editions, back and forth

Is there any way to provide the "adequate" commands for each relevant case, so as to avoid these back_and_forth editions?

[1] [2] [3]

Ady (talk) 16:08, 27 September 2016 (UTC)

And here we go, again. Ady (talk) 05:20, 15 October 2016 (UTC)
I see 4 options here:
1. Ban the users in question for edit warring instead of using the talk page, or temporarily lock the page
2. Remove the section and link to some OS X resource instead
3. Use a different tool like cat
4. Mention that you might have to use 1M if 1m doesn't work, or vice-versa.
I'm inclined to go with 2.
-- Alad (talk) 08:28, 15 October 2016 (UTC)
Even before I started this Talk section, I already thought I would go with your option 4. I think it should stop the "back and forth", it would be a simple, short and clear addition to the current text, it would respect users that have invested their time to add the whole set of instructions, and the general steps in the section would remain valid, without depending on third-party sites / sources. Ady (talk) 09:53, 15 October 2016 (UTC)
Almax (talk) 12:23, 15 October 2016 (UTC)
Guilty as charged. My guess for why back and forth is happening: Linux folks change the block size multiplier to uppercase M when they see it in the wiki (correct syntax for Linux dd) and Mac folks change it to lowercase m (correct syntax for Mac OS/BSD dd; only accepts b, k, m, g, w). The only correct edit for Mac OS is lowercase m, but how to prevent well-meant "corrections"? I support option 4-ish: Add a Note: explicitly explaining that the args are different between the systems and/or explain it when showing the actual dd command. I arrogantly rewrote the whole section yesterday because I found it to be oddly worded, containing redundant steps and not including info about the device's file system not being recognized by Mac OS after writing the arch*.iso to it. I hope the edit is acceptable. I'm new in the Arch community but no newcomer to Macs or Linux.


Part of the problem is that some users would "extend" certain concepts and/or commands from one OS to another, but other users would interpret this extension differently, or even perform it differently.
For instance, some users would assume that some command in Linux can be executed exactly the same in BSD, with the same expectations / results. Others might assume the same behavior when going from Mac OS X (or whichever similar name would be valid/current at each time) to BSD.
For example, User:Agentxp22 performed at least one of these editions (I re-post here the link for convenience), [4] with a summary note:

1m > 1M because BSD dd uses upper case M for MB block size

I am not getting into the matter of which command is correct (and which is not) for each OS. This Talk section is not about that.
I think we should not "fight" such assumptions, because (too) many users have them, and "policing" each edition (not just in one article) is time-consuming and prone to mistakes. Instead, IMO a more practical solution is to just add a relevant minimal paragraph/sentence/note, as suggested by the 4rth alternative solution proposed by Alad, at least in this particular case.
As I said, it should stop the "back and forth" editions, and clarify the procedure/command while avoiding assumptions. Ady (talk) 14:55, 15 October 2016 (UTC)

And now we have [5]. Ady (talk) 15:00, 8 August 2019 (UTC)

If no one can find out the correct unit specification for macOS, just remove the bs parameter entirely and let the macOS users write 512 bytes at a time. -- nl6720 (talk)


Ady, that was me and now I realize which version of dd I used, the GNU one. We should use the syntax for the default version that comes with macOS, which is the FreeBSD version, and optionally add a comment to specify for those that use the coreutils version.
From the GNU man page:
    BLOCKS and BYTES may be followed by the following multiplicative suffixes: c
    =1, w =2, b =512, kB =1000, K =1024, MB =1000*1000, M =1024*1024, xM =M GB
    =1000*1000*1000, G =1024*1024*1024, and so on for T, P, E, Z, Y. 
From the FreeBSD man page:
    Where sizes or speed are specified, a decimal, octal, or hexadecimal num-
    ber of bytes is expected.	If the number ends with	a ``b, ``k,	``m,
    ``g, ``t, ``p, or ``w, the	number is multiplied by	512, 1024
    (1K), 1048576 (1M), 1073741824 (1G), 1099511627776	(1T), 1125899906842624
    (1P) or the number	of bytes in an integer,	respectively.  Two or more
    numbers may be separated by an ``x to indicate a	product.
Goetzc (talk) 17:21, 11 August 2019 (UTC)
I should clarify, again. As I mentioned before, this specific Talk section is not really about which command is correct for each OS; that's a "technical" matter, and in _theory_ it should be quite "easy" to find out the "correct" one.
The problem is that, once in awhile, someone decides to "correct" the command in the wiki while betting mom's life that the "corrected" command is truly the "correct" one. And then comes another editor with similar bets/claims but with a slightly different command (e.g. "m" vs. "M"), until a third one steps in (or the first one comes back).
You know, theory vs. practice; relatively few editors would check a Talk page or open a new discussion before performing an edition, especially when their own "bets" are so "absolutely the winner" ones, "no doubt".
Then comes some fourth editor, suggesting to eliminate the parts of the command that might generate some conflict while being (seemingly) non-essential, until a fifth editor re-adds the same conflicting option/parameter/argument to the command in question.
There are also those that suggest that "we should just remove everything", even the parts that are relevant and useful. That's until someone else invests the time to re-add something similar, yet again.
(Note: I'm not trying to be rude in any way to anyone; I'm trying to clarify a point.) My point is: let's try to add a simple sentence/note that would clarify the matter and lead users (and potential wiki editors) to attempt one command, and if it happens to fail, then try the other, while "leaving the text alone" (instead of this "back_and_forth"). Or (and) link to relevant sources, or (and)... In a certain sense, keep it as simple as possible and as complex as required while being useful in a pragmatic way.
So, now that I have rephrased the matter (hopefully in a clearer way/scope/perspective), I guess it is time for me to ask... Which is the adequate way/text/format/information that would achieve this balance while reducing/stopping the "back_and_forth" editions?
Reminder: the command in question is part of a section that (at the time of this writing) is named "In macOS". Ady (talk) 20:18, 11 August 2019 (UTC)
How about adding something like this:
Note: The following command uses macOS dd. The block size is specified differently than for GNU coreutils' dd.
-- nl6720 (talk) 17:48, 10 February 2020 (UTC)
I added a comment in the page source to future editors not to change it to 'M', and removed the dispute tag ('m' is absolutely correct for the default 'dd' on macOS). Hopefully this is enough. ColdPie (talk) 14:06, 4 May 2020 (UTC)
After someone reverted that, I added a note instead of an HTML comment. Hopefully instead of reverting, future editors will make improvements. ColdPie (talk) 13:52, 5 May 2020 (UTC)
Special:Diff/645080 once again made the section unclear :/ -- nl6720 (talk) 07:21, 26 February 2021 (UTC)

Using manual formatting In GNU/Linux

Is it truth that fs on partition _must_ be FAT32? For uefi it is guaranteed to work, but for some firmware implementations it is also possible to use other fs types as I know. For legacy it could be also ext2/3/4 I guess (even syslinux is called extlinux). So I think it is better to say something like "for better compatibility" instead of "must" here.

When copying files from mounted iso, maybe add exclude arguments to the command? For example, we could exclude isolinux folder.

Actually, in instructions here: https://gitlab.archlinux.org/archlinux/archiso/blob/master/docs/README.transfer they mention another FSs and use excludes in extraction command. Ashark (talk) 17:47, 25 April 2019 (UTC)

dd-ed ISO to USB device + new partition for user data transfer - without failing to boot

If for whatever reason, user does not want to use either of USB_flash_installation_media#Using_manual_formatting (because it requires FAT32 for example) or USB_flash_installation_media#Using_a_multiboot_USB_drive, then here's a way to still be able boot (without failing into rescue shell) in the case when you did dd the image to the usb (via USB_flash_installation_media#Using_dd) and took the additional step to make a new partition for extra data that you wanted to have available during installation for example.

Due to the way systemd identifies by-label mounts for isohybrid ISOs (see issue in https://github.com/systemd/systemd/issues/12409 ) of which archlinux*.iso is one, you have to use kernel cmdline archisodevice=/dev/disk/by-partuuid/ISOUUID-01 where ISOUUID is what you get when you execute blkid -p archlinux-2019.04.01-x86_64.iso seen in its output as PTUUID="377032cf" (don't forget to add that -01 which means the first partition (which is the 600M iso one)), for example: archisodevice=/dev/disk/by-partuuid/377032cf-01. For other ways to pass archisodevice to kernel cmdline, see how it's being done inside the section USB_flash_installation_media#Using_manual_formatting. Note: I've only tested grub boot (so non-UEFI) with this method and I was thus able to mount my new partition this way. Without this by-partuuid trick, booting the iso from the USB device will fail at mounting airootfs, then if workaround it by just giving your new partition a label, it will boot ok but you won't be able to mount this new partition (details in the aforementioned systemd issue on github). Howaboutsynergy (talk) 22:35, 13 May 2019 (UTC)

Rufus ISO Image Mode no longer works

From the looks of it DD Image Mode is now required. I've only tested it with UEFI. Tonij (talk) 15:15, 7 March 2020 (UTC)

GUI application for multiple platforms - USBImager

Hello Folks,

I am planing to add USBImager as GUI application for below platforms.

-Linux

- Mac

- Windows

—This unsigned comment is by Renegade (talk) 10:18, 3 April 2021‎. Please sign your posts with ~~~~!

Feel free to add it. -- nl6720 (talk) 09:06, 15 April 2021 (UTC)