Talk:USB flash installation media

From ArchWiki
Revision as of 14:52, 7 January 2017 by Pineman13 (talk | contribs)
Jump to navigation Jump to search

About making the installation media without overwriting

I'm not totally sure if I misunderstood something, but I had to change the path of the entries of the *.cfg files. For instance:

  • Create one partition entry in GPT (of type "ef00")
  • Create a FAT32 filesystem on such partition replacing $LABEL with the one that corresponds to the archiso label. In case you have an archlinux-2017.03.01.iso, the label is ARCH_201703
  • Mount target filesystem. In this example we mount it on /mnt/esp
  • Extract ISO image on target filesystem.
  • Unmount target filesystem.
# gdisk /dev/sdX
# mkfs.fat -F 32 -n $LABEL /dev/sdXn
# mount /dev/sdXn /mnt/esp
# bsdtar -x --exclude=isolinux/ --exclude=EFI/archiso/ --exclude=arch/boot/syslinux/ -f archlinux-2017.03.01.iso -C /mnt/esp
# umount /mnt/esp

INCLUDE boot/syslinux/archiso_sys.cfg


INCLUDE syslinux/archiso_sys.cfg

It was the only way it worked with the unofficial ISO x86_64 image of march 13th, 2012. Looks like the syslinux command described in the page doesn't get the path as it should.

I edited all of the .cfg files, but probably only editing this ones should have been enough:

archiso.cfg archiso_head.cfg archiso_sys_inc.cfg

I hope it could be useful to somebody, because I spend some time with this (I even thought that was a problem with the hardware). I think it could be possible to make a simple script (or give some command lines) to patch the files once they are copied into the USB and run syslinux.

Thanks !!

I cannot find that this is still relevant to the article as it now exists. I am striking it out as it looks like it could be good to remove. AdamT (talk) 10:33, 8 August 2013 (UTC)
Just lunched from USB drive without UUID(don't have any idea why it didn't have one). Solution was to change label to appropriate in loader/entries/archiso-x86_64.conf. Not sure weither this should be added to article.

--Versusvoid (talk) 16:00, 18 August 2013 (UTC)

Versusvoid, I would love to adapt the article but I cannot follow your description above. Looking at that section, it may be out dated. If you are watching this please elaborate when you have time. Thanks, AdamT (Talk) 08:32, 23 August 2013 (UTC)
Ok. For some reason ubuntu did not see USB UUID. So blkid -o value -s UUID /dev/sdx1 were returning empty string. The solution was:
$ sed -i "s|label=ARCH_.*|label=$(blkid -o value -s LABEL /dev/sdx1)|" loader/entries/archiso-x86_64.conf
in the USB mounting directory. --Versusvoid (talk) 08:05, 31 August 2013 (UTC)
It seems that sometimes you have to get the UUID by running the command as root. --Rongmu (talk) 11:23, 3 October 2013 (UTC)
Thank you for the follow-up. My semester started so I have not had much of a chance to help with the Arch Wiki of late. I am removing the strike from this section until I can take a closer look and update the article. AdamT (Talk) 02:02, 16 November 2013 (UTC)

Making an UEFI/BIOS ISO should be KISS

Following up on the discussion in the the bbs, I disagree that this edit should be kept as the first thing users see when hitting this page. I based this opinion on my feeling that the instructions are too many steps and, arguably too vague. Example, this method is 7 steps (depending on how you count) and requires that user read linked articles (i.e syslinux install and modifying master boot records). In contrast, the dd method is simple (KISS principal) and is both implemented and documented as pointed out by one of our developers in the aforementioned bbs thread.

I think we should at least start the article with the KISS method and this edit down the page. Graysky (talk) 10:30, 29 November 2013 (UTC)

Well, I have been keeping an eye on this since the.ridikulus.rat's extensive edits on and after 2013-11-20. I have kept quiet because the maintainers and admins seemed to accept the changes. However, from the start, I disagreed along lines similar to what Graysky has mentioned above.
I did not, and do not, understand why the Arch Wiki should be recommending a specific method without any references or firm reasoning. Instead the.ridikulus.rat seemed to be prescribing a method that was believed was superior based on their own preferences. This method is slightly more complicated than writing the image directly with dd, but it does keep the drive usable for data storage.
Please note this quote from the BBS thread Graysky linked above (emphasis mine): It took me some time to read through the syslinux docs and other blog [sic] to understand the syslinux installation process under Windows, and I don't appreciate you simply removing the entire part that I thought out and typed for the sake of the community. I just realized I completely mistook what the.ridikulus.rat meant to say here. Please disregard.
While I do think the technical information that the.ridikulus.rat provided was needed in general, and I had in fact proposed something similar shortly before, the extensive reordering of the page and the subjective recommendations does not seem in keeping with the ideals and established processes of the Arch Wiki as I understand them.
Further, referencing the quote above, egos definitely seem to be coming into play in a place where they should not matter. The sake of the community is what matters here, not individual investments. See also The Arch Way.
Moving forward, as of this writing, the changes that Teateawhy has been making seem to be in keeping with the practices put forth in the Arch Wiki's documentation while also mitigating the subjective recommendation that was put forth during the.ridikulus.rat's changes.
As a fairly new contributor, one aspect I have not been able to determine from reading over the Arch Wiki's documentation is when and how to make recommendations. As such, and keeping an eye on the bigger picture here, I would like to take this opportunity to suggest something be added to Help:Style or elsewhere regarding when and how recommendations and suggestions are justified. There seems to be a willingness for some contributors to make claims without references or supporting statements. Similar to Wikipedia's "reference needed" template, a flag or at least cohesive policy may be warranted for the Arch Wiki and it may help prevent situations like this in the future! : )
Cheers all, AdamT (Talk) 03:17, 30 November 2013 (UTC)
I agree that dd is the easiest method, but I would like to use the manual method in order to use a USB drive also as data storage. I'm not new to Linux by a long shot, but the text is so garbled up that I can't successfully follow the manual method... User:Jrg2718, 07 May 2014

Side notes

I don't want to hijack the main discussion, but I'd like to answer a couple of AdamT's observations:

  • please do not assume that admins and maintainers can follow everything that happens on the wiki; if you have something to say about somebody else's edits just do it ;)
  • I'd be glad to add some guidelines about recommendations to the style guide; the problem is that it's still very subjective to distinguish between what is a justified recommendation and what is a personal recommendation... Technically The.ridikulus.rat did justify his suggestion. If you have an idea for the wording of an effective style guide, please propose it here or in Help talk:Style, but I think that most of these cases will have to be solved in discussion pages like it's happened here.

-- Kynikos (talk) 01:34, 1 December 2013 (UTC)

Other Methods for BIOS systems

Do we really need these provided that dd just works? I suggest deletion of them as outdated. SkyBon (talk) 15:12, 28 January 2014 (UTC)

Are they outdated, i.e. you're sure they don't work? Otherwise I don't see the reason to delete them, we let users choose their preferred method. -- Kynikos (talk) 03:55, 29 January 2014 (UTC)

Graphical dd methods for Linux and Windows

Hi from Fedoraland, Arch folks! I just finished updating the Fedora wiki page for doing USB writing - with some references from the rather good SUSE pages on the same topic - main page, Windows, OS X. I think Arch, Fedora and SUSE all prefer a 'dd' style writing method, so we can take cues from each other here. The Fedora and SUSE pages both now feature ways to do a dd-style write on Windows and OS X, and the Fedora page has a rather convenient method for doing a dd-style write from GNOME - perhaps Arch might like to pick these up too? Could be handy for your users. I tested the recommended Windows tools (SUSE's Studio writer, and rawrite32) on a Windows laptop, and they both clearly did a clean direct write of the image, with Fedora's complex disk label setup to ensure BIOS, UEFI and Mac boot all work kept intact. i.e., they don't screw crap up like unetbootin or that Ubuntu tool does.

I suppose we could even think about providing a unified set of instructions / tools somewhere - there's quite a bit of redundancy going on at the moment. Thanks! AdamW (talk) 19:06, 30 January 2014 (UTC)

Hi AdamW, thank you for your work and for trying to involve our community! For the moment I've referenced the Fedora and openSUSE articles from our See also section, however let's keep this discussion open to see if somebody wants to adapt the methods we're missing for Arch on this article. We may even add some Expansion templates where needed, as reminders and to encourage contributions. In alternative, since in general instead of duplicating external sources we prefer to just provide links to them if they're well maintained, we may just add more specific links to the Fedora article in the proper sections.
Of course it would be great if we managed to reduce the redundancy among our articles with a unified article "somewhere", but the problem would be how to make it easy for our respective wiki users to edit such article without forcing them to register on another wiki (which clearly would instead have the effect of discouraging contributions). Maybe we could keep part of our articles synchronized with a bot?
-- Kynikos (talk) 04:25, 1 February 2014 (UTC)

dd for windows

\\.\ notation is a Microsoft construct. It addresses a device in the DOS namespace. The documentation is here:

\\? is a dd construct. It means address a device in the NT native namespace.

Not all devices are available in the DOS namespace and the rules vary between OS versions and partition type codes etc. Jnewbigin (talk) 01:32, 12 December 2014 (UTC)

I tested the \\.\ notation with Windows 10 1607 (OS Build 14393.222) and cygwin but it did not work. Instead I used /dev/sdx and it worked perfect. Hncr (talk) 23:30, 13 October 2016 (UTC)

Using Universal USB Installer

This bit of advice doesn't currently work. The media creates fine but when you try to boot, you get exactly into this story. All3fox (talk) 07:41, 13 April 2014 (UTC)

The problem and bbs thread with explanation is referenced in USB Flash Installation Media#Using Universal USB Installer meanwhile. Closing this. Thanks for reporting! --Indigo (talk) 17:55, 16 August 2014 (UTC)
Re-opening, wrt the following edition(s) .
I am confused by this specific edition(s). AFAIK, and at least at the time I am writing this (2014Sep), UUI does not support booting on UEFI systems. To be clear, I mean that UUI is aimed to prepare a USB flash drive to initially boot by means of Syslinux under BIOS (with the caveats already mentioned in the wiki article), and UUI does not know how to boot/use syslinux.efi (nor any other UEFI bootloader). Am I misinformed?
I am not saying that a USB flash drive, after using UUI, is incapable of booting UEFI hardware, but it is not UUI that makes it happen. So mentioning UEFI in a Note about UUI is slightly confusing (to me).
I don't want to undo the editions before I am sure I am not misunderstanding something. So, am I? Ady (talk) 22:11, 16 September 2014 (UTC)
I wrote the UUI section and came here to ask the same question: this section is about making an Arch boot USB using windows, and is not specific to EFI-bootable USB disks. The goal is just to bootstrap Arch on the system. Once Arch is booted you can set up your permanent storage to boot any way you like. In particular, I have used a UUI-created boot disk to set up a UEFI system many times. -- Pgoetz (talk) 11:05, 5 January 2015 (UTC)
To test, I used UUI to create a ARCH_201501 USB boot disk, and then proceeded to use it to EFI boot and install a new server. I'm going to remove the Inaccurate warning, since it is itself inaccurate.

Syslinux note

-- Moved from Help:Style, as the discussion shifted back to this article. -- Alad (talk) 19:46, 22 September 2014 (UTC)

@Ady (reopening): first of all I want to make clear that I didn't write the "own[ing]" argument in an accusing way, it was only meant to explain my point of view, I should have added a smiley at the end, I'll do it now :) More important, you said we "derail[ed] from the initial goal", but actually I asked you some questions that were very relevant and specific to your edit, and you haven't answered, so I'll paste them here again: Where exactly is that note inaccurate? Because usually when you feel some deeper explanations are out of place in some article it's because that article's topic is quite different, so the details should belong somewhere else: is it something regarding syslinux? Because we have an entire article for it, maybe you could add what's needed there and simply link there from the note? -- Kynikos (talk) 03:49, 17 August 2014 (UTC)

@Kynikos, Since the discussion was moved, the focus changed from the more-technical matter to whether using hidden text was/is appropriate (in this particular case). Thus, I thought that perhaps answering those questions here would have been considered off-topic.
The referenced Note says "The above step installs Syslinux's ldlinux.sys to the VBR of the USB partition, sets the partition as "active/boot" in the MBR partition table and writes the MBR boot code to the 1st 440-byte boot code region of the USB.". As you can see, this Note is not part of a practical step (and therefore, a slight inaccuracy is likely not going to affect a typical reader following the instructions). Yet, there is a purpose for this Note.
Users kept asking (including to upstream Syslinux) whether they can skip part of the steps, and just copy over ldlinux.sys (for example, to either install or update SYSLINUX in their USB drives). Moreover, some users didn't even ask; they simply (over)wrote this file and they complained that SYSLINUX (and/or Arch) fails to boot. So the Note is there to prevent this type of behavior / confusion / repeated waste of time. This explains why deleting the referenced Note would not be wise.
About the minor inaccuracy... What the SYSLINUX installer does (when performing the specific steps exactly as described in the relevant section of the article) is to:
  • copy ldlinux.{sys,c32};
  • patch the VBR so it can point to the correct block in the device (so ldlinux.sys can be found);
  • set the partition as "active/boot" in the partition table;
  • write the MBR boot code to the 1st 440-byte boot code region of the USB.
I want to clarify that the above description is only relevant for the specific section we are referring to. The SYSLINUX/EXTLINUX installers perform different tasks according to different cases.
So, there is a minor (technical) background inaccuracy in the Note.
Considering the goal of this Note (as I already described it above) in the context of this particular section of this particular article, and considering that the practical steps provided in the article are not affected by this minor inaccuracy, I asked my self (at the time I performed other editions on the article) whether I should also edit the Note. Would I be actually helping a typical reader that is interested in following the steps? With the objective to balance between accuracy and clarity for the typical reader, I decided not to over-complicate the Note with excessive details that in practical terms only would make the section longer without improving the outcome.
Adding a link to the Syslinux article would not improve the practical experience / steps for the user in this case. If anyone wants to find more technical information, they don't need yet another link from that same article to the same Syslinux page. In this article, the typical reader is (and/or should better be) likely focused on the practical steps.
Thus, my "hint" was "This note is not %100 technically accurate, but it is "accurate-enough" so to be able to perform the task.". For the typical reader trying to follow the practical steps, the procedure is accurate and he will complete the task successfully, so adding an "Accuracy" box would be counterproductive.
Generally speaking, I think there are cases where adding 'comments' (aka hidden text) is adequate, but I am perfectly fine following the guidelines in Help:Style. Lahwaacz's edition formatted the "hint" as "Accuracy" box, going exactly in the opposite direction of what the hint itself was saying (or at least, against the original intention of the comment). This was not beneficial for users, so I deleted the box.
Slightly long to read (apologies), but I hope this clarifies the matter. Ady (talk) 10:45, 17 August 2014 (UTC)
I see, thanks for explaining, then why not make explicit the reason why the note is there (KISS), like:
  • Make sure to run the above command in full, which installs Syslinux's ldlinux.sys to the VBR of the USB partition, sets the partition as "active/boot" in the MBR partition table and writes the MBR boot code to the 1st 440-byte boot code region of the USB. Only copying the file, for example in an attempt to update an existing Syslinux installation, will result in an unbootable device.
  • ...
This way nobody will see the note as useless and delete it.
-- Kynikos (talk) 13:50, 19 August 2014 (UTC)

"... and install the MBR or GPT partition table to the USB device as described in the page mentioned."

I am somewhat clueless about what page this instruction in USB_flash_installation_media#Using_manual_formatting is referring to, can anyone clear that up, what is "the page mentioned"? There is no further mentioning of GPT in the article. Also, not clear why it should be necessary to create a partition table on a USB device, which obviously is partitioned already (else, I wouldn't be able to manipulate any files on it). --Johannes Rohr (talk) 08:40, 5 February 2015 (UTC)

The previous link leads to Syslinux#Manual_install, have you looked there? -- Lahwaacz (talk) 09:46, 5 February 2015 (UTC)
Also, not clear why it should be necessary to create a partition table on a USB device, which obviously is partitioned already
I think this has to do with UEFI vs. BIOS booting. In the case of the more modern UEFI, you have a special FAT32-formatted boot partition which is labeled EF00 (so there are 2 partitions on the disk). If you're just going to create a BIOS boot USB, then you only need a single partition. Pgoetz (talk) 09:44, 5 February 2015 (UTC)

Repeated editions, back and forth

Is there any way to provide the "adequate" commands for each relevant case, so as to avoid these back_and_forth editions?

[1] [2] [3]

Ady (talk) 16:08, 27 September 2016 (UTC)

And here we go, again. Ady (talk) 05:20, 15 October 2016 (UTC)
I see 4 options here:
1. Ban the users in question for edit warring instead of using the talk page, or temporarily lock the page
2. Remove the section and link to some OS X resource instead
3. Use a different tool like cat
4. Mention that you might have to use 1M if 1m doesn't work, or vice-versa.
I'm inclined to go with 2.
-- Alad (talk) 08:28, 15 October 2016 (UTC)
Even before I started this Talk section, I already thought I would go with your option 4. I think it should stop the "back and forth", it would be a simple, short and clear addition to the current text, it would respect users that have invested their time to add the whole set of instructions, and the general steps in the section would remain valid, without depending on third-party sites / sources. Ady (talk) 09:53, 15 October 2016 (UTC)
Almax (talk) 12:23, 15 October 2016 (UTC)
Guilty as charged. My guess for why back and forth is happening: Linux folks change the block size multiplier to uppercase M when they see it in the wiki (correct syntax for Linux dd) and Mac folks change it to lowercase m (correct syntax for Mac OS/BSD dd; only accepts b, k, m, g, w). The only correct edit for Mac OS is lowercase m, but how to prevent well-meant "corrections"? I support option 4-ish: Add a Note: explicitly explaining that the args are different between the systems and/or explain it when showing the actual dd command. I arrogantly rewrote the whole section yesterday because I found it to be oddly worded, containing redundant steps and not including info about the device's file system not being recognized by Mac OS after writing the arch*.iso to it. I hope the edit is acceptable. I'm new in the Arch community but no newcomer to Macs or Linux.

Part of the problem is that some users would "extend" certain concepts and/or commands from one OS to another, but other users would interpret this extension differently, or even perform it differently.
For instance, some users would assume that some command in Linux can be executed exactly the same in BSD, with the same expectations / results. Others might assume the same behavior when going from Mac OS X (or whichever similar name would be valid/current at each time) to BSD.
For example, User:Agentxp22 performed at least one of these editions (I re-post here the link for convenience), [4] with a summary note:

1m > 1M because BSD dd uses upper case M for MB block size

I am not getting into the matter of which command is correct (and which is not) for each OS. This Talk section is not about that.
I think we should not "fight" such assumptions, because (too) many users have them, and "policing" each edition (not just in one article) is time-consuming and prone to mistakes. Instead, IMO a more practical solution is to just add a relevant minimal paragraph/sentence/note, as suggested by the 4rth alternative solution proposed by Alad, at least in this particular case.
As I said, it should stop the "back and forth" editions, and clarify the procedure/command while avoiding assumptions. Ady (talk) 14:55, 15 October 2016 (UTC)

Universal replacement for "dd"

I've used "ddrescue" command in the past to create working media for different distributions. It has a "progress bar" built-in as well as error reporting. Should this be the default recommendation now instead of "mere dd"?

—This unsigned comment is by Beoldhin (talk) 20:06, 28 December 2016‎. Please sign your posts with ~~~~!

The dd commands on this page include status=progress, which shows the progress. I don't see a reason to add yet another dependency to be installed. -- Lahwaacz (talk) 20:40, 28 December 2016 (UTC)

Add section 'Other Methods for UEFI systems'

There's a certain amount of EFI devices that need editable(manually formatted) partitions. Such as 32-bit UEFI devices that need custom .efi files added simply in order to boot. How about adding that as a section, since we already have one for BIOS?

I have included the text that I'd like to add to the article(it's commented out, edit this post to see it). Pineman13 (talk) 14:51, 7 January 2017 (UTC)