User:Andy Crowd

From ArchWiki


My achievements

My packages in AUR: list them.

My sandbox pages: A list of other sub/fun

Fast Notes & current problems that need to be solved

Current links as tmp-memo I need to use
  • Download Chinese fonts
A="$(curl http://www.clearchinese.com/resources/fonts.htm | grep -i ttf | sed 's/[^*]*href=\"//g' | cut -d\" -f1)"
  • Show list of unique category names. Could simplify and added as option --lc in lsdesktopf to search only in /usr/share/applications/ and $HOME/.local/share/applications/.
  • Now available new updated version in AUR4 lsdskAUR. Link to forum forum.
    • Preparing info to make an extended version of lsdsk script that will support command line. It will be named as lsdesktopfitems, Link to read multiple items from command line.
  • QEMU - examples to test
  • Future project: Live CD + auto-configuration of pulseaudio + detect user-names in Windows C:\users\ folder and create them in Linux by using C:\users\user_name as a home path.

[1], [2] or [3]

  • To make DVD video work good forum, libs need to be installed: libdvdnav libdvdread libdvdcss

Fix for MPEG: No audio stream found -> no sound.

  • A4 = 210x297 mm - command line for sane
scanimage -x 120 -y 210 --resolution 300 --mode Color --format=tiff >image2.tiff
  • Sakis3g and QT bug fix: export QT_X11_NO_MITSHM=1 before X start

URL extraction new version 2gisAUR 

$ curl "http://info.2gis.ru/moscow/products/download#skachat-kartu-na-komputer&linux" | awk -Fzip '{if(match($0,"2GISShell") != 0){AA=substr($2,index($2,"http"));if(match(AA,"http") != 0)print AA"zip"}}'

Convert media scripts

My scripts for conversion of media files

m4a to ogg 

m4a_to_ogg.sh
#!/bin/bash
for I in "$HOME"/Media/Music/*.m4a;do PP=${I##*/};
if [ -d  "$HOME/Media/Music/OGG" ]; then
####
ffmpeg -i "$I" -acodec vorbis -strict -2 -ac 2 "$HOME/Media/Music/OGG/${PP/.m4a/.ogg}" ;
####
IR=$(du "$HOME/Media/Music/OGG/${PP/.m4a/.ogg}" | awk '{print $1}')
if [ "$IR"  != "0"  ]; then 
if [ -d "$HOME/Media/Music/Converted" ]; then 
mv -vi "$I" "$HOME/Media/Music/Converted";
else 
echo The '"$HOME/Media/Music/Converted"' folder is missing
break
fi
else 
echo Something gone wrong size of converted "$PP"  file is 0
break;
fi;
else echo Path doesn"'"t exist: '"$HOME/Media/Music/OGG/"'
break
fi
done

Auto-gen configuration files

Perfect to use them on a Live CD

Conky

All moved to github

HDDTemp

Moved to github

TMP for before deletion

. 
           ************************************************
           **** Your system is too SLOW to play this!  ****
           ************************************************

Possible reasons, problems, workarounds:
- Most common: broken/buggy _audio_ driver
  - Try -ao sdl or use the OSS emulation of ALSA.
  - Experiment with different values for -autosync, 30 is a good start.
- Slow video output
  - Try a different -vo driver (-vo help for a list) or try -framedrop!
- Slow CPU
  - Don't try to play a big DVD/DivX on a slow CPU! Try some of the lavdopts,
    e.g. -vfm ffmpeg -lavdopts lowres=1:fast:skiploopfilter=all.
- Broken file
  - Try various combinations of -nobps -ni -forceidx -mc 0.
- Slow media (NFS/SMB mounts, DVD, VCD etc)
  - Try -cache 8192.
- Are you using -cache to play a non-interleaved AVI file?
  - Try -nocache.
Read DOCS/HTML/en/video.html for tuning/speedup tips.
If none of this helps you, read DOCS/HTML/en/bugreports.html.

Restore original file names by using backup file with checksums and comparing with List only unique files by checksum generated file. 

awk -F"|" -v W="$(cat compmd5_new.tmp)" '//{split(W,Z," ");
for(i in Z)if(index(i/2,".") != 0){if(Z[i] == $4){F=Z[i+1];gsub(/[^\/*]*\//,"",F);
print $1"|"$2"|"$3"|"F};}}' compmd5_1.tmp

This is the same as above but will also handle filenames with spaces correctly 

awk -F"|" -v W="$(cat compmd5_new.tmp|awk '{print "|"$1"|"substr($0,index($0," "))}')" '//{split(W,Z,"|");for(i in Z)if(index(i/2,".") == 0){if(Z[i] == $4){F=Z[i+1];gsub(/[^\/*]*\//,"",F);print $1"|"$2"|"$3"|"F};}}' checksums.list | grep -v ^$

Link to forum where I was looking for help: linuxforum

Populate array by image extensions, may be not work correctly if some part of extension exist in the list, e.g. h extension will be found in html. 

RR="jpg gif";
QQ=($(awk -F'|' -v KK="$RR" '{SS=$3;gsub(/[^*\.]*\./,"",SS);if ( index(KK,SS) != 0 ) print $3}' checksums.list)) 
for (( i=0;i <= ${#QQ[@]};i++ ));do if [ ! -z  ${QQ[i]} ];then echo ${QQ[i]};fi;done

This will clean up special symbols, sort restored names, add a number to the duplicate names. 

#!/bin/bash
if [ ! -z "$1" ];then
  awk -F"|" '{B=$2;
   gsub(/\(/,"",B);gsub(/\)/,"",B);
   gsub(/!/,"",B); gsub(/?/,"",B);
   gsub(/\[/,"",B);gsub(/\]/,"",B);
   gsub(/{/,"",B); gsub(/}/,"",B);
   gsub(/&/,"",B); gsub(/=/,"",B);
   gsub(/\^/,"",B);gsub(/~/,"",B);
   gsub(" ","",B) ;gsub(/#/,"",B);
   gsub(/\"/,"",B);gsub(/;/,"",B);
   gsub(/\\/,"",B);gsub(/\//,"",B);
   sub(/-*/,"",B); sub(/+*/,"",B);
   print $1" | "B" | "$3}' "$1" | \
  sort --field-separator=\| -s -d -k 2 | \
awk -F'|' '{B=$2;Count[B]++;sub(/ */,"",B);if( Count[$2] == 1 ){print $1" | "B" | "$3}else{print $1" | "Count[$2]-1"¤"B" | "$3} }'
else echo 'Path to file is missing!'
fi

I will probably rewrite my post_rec_scripts to use checksum file, already sorted file names.

With grep and awk commands populate the array

This way of populating an array is many times faster as with a while command but has limitations that might cause errors. A common way of populating an array as in this example causes problems due using space between words as a separator and a file names that contains them will not be restored and errors will be shown. A $SearchFor variable is more intuitive to edit then if all patters are in the same line with grep. 

SearchFor="-e compressed -e archive";
ArrayOfFiles=($(grep -i $SearchFor   info-mime-size-db.txt | awk -F'|' '{print $1  }'));

Without grep you have to use if inside of gawk command and add patterns. Suitable if file with data is a really a very big and you can chose in which part of string you want search compared to grep that uses a whole string. 

ArrayOfFiles=($( gawk -F '|' '{if ($3 ~ "image/jpeg" || $3 ~ "image/gif" || $3 ~ "image/png") print $1 }' info-mime-size-db.txt))

You can find out which of recovered files contains spaces in their names and save information about them in a file for future use. 

$ find . -type f -name "* *" >> filenames-with-spaces.txt
$ gawk -F'|' '{if ($1 ~ " ") print $1 }' info-mime-size-db.txt >> filenames-with-spaces.txt

Calculate duplicate files with awk

Sort out duplicates, tests with any check summer 

md5sum f*.pdf | awk '// {Count[$1]++;CNames[I++]=$1}END{ for (i in Count) {if( Count[i] > 1 )print Count[i]" "CNames[A++];}}'

Note about misc

Note: Better to make a python script to speed up all of this... it has libs for management of images. Soon...month or more until I will be will be done with all my additional ideas as a command line support(srt,img,...) and much more in a single file, in a python3 version of p.rec.tsk. and related scripts, will be published on a git hub after that.

github-wiki - will be home of my python3 scripts.

Original name only from photorec recovered path

A lot of cuts but no need to use an external program/utility and can be used with loops(while/for): 

  AA='./recup_dir.1/f864563104_wmcloc_kmon-0.1.0.tar.gz';
  ZZ=${AA/*\//};
  BB="${ZZ/_*/}_";
echo ${ZZ/$BB/}

Cuts away generated names by photorec from original, cannot be used with external loops: 

$ awk -F'|' '{AA=$1;sub(/^.*\//,"",AA);if ( AA ~ "_") {BB=index(AA,"_")+1; print substr(AA,BB )}  }' info-mime-size-db.txt

All in one

Used folder and files auto create test ground section for aa.txt in the example below.

By using IFS bash script special standard variable to change separator it is possible fill in an array with strings that contains spaces. Works perfect, will create test pattern 

./recup_dir.1010/f872681448_wmavgload-0.6.1.tar.gz | OName= wmavgload-0.6.1.tar.gz |
./recup_dir.1010/f872688972.txt| FName= f872688972.txt |

Full path with filename|Destination name, cut to orig if exist in it|

In array it will use step by two with | as a separator, e.g. 

IFS="|"
C="0";
A=ArrayItem[C]
B=ArrayItem[C+1]
C=$((C+2))

#!/bin/bash

awk -F'|'  '//{AA=$1;
sub(/^.*\//,"",AA);
            BB=AA;
          if ( AA ~ "_") {
             GUline=index(AA,"_")+1;
             OName=substr(AA,GUline );
print $1 " | OName= " OName " |"
}
else {SIName=index(AA," ");
if (SIName) { SWName=AA;
  print $1 "| SWName= " SWName " |"
        }
  print $1 "| FName= " BB " |"
 };
 }'  info-mime-size-db.txt
  • Can cut path to show only base filename.
  • Can cut generated name by photorec from original.
  • If generated filename doesn't contain original as part of it then output generated into array.
  • Can fill in array with strings that contain spaces by setting up and use a new separator with help of IFS special bash variable.

See also: internal bash variables.

Base name only

Example base name only: 

$ awk -F'|' '{print $1}' info-mime-size-db.txt | sed 's/[^*/]*\///g'

With printf will show errors like not enough arguments to satisfy format string if variable contains some of symbols that it uses as expressions e.g %: 

$ awk -F'|' '{AA=$1;sub(/^.*\//,"",AA);printf AA "\n"}' info-mime-size-db.txt

With print will show no errors because it doesn't use expressions: 

$ awk -F'|' '{AA=$1;sub(/^.*\//,"",AA);print AA }' info-mime-size-db.txt

See also: awk manual

Two more alternatives to fill in an array with data without using of grep: 

gawk -F '|' '{if ($1 ~ "bmp" || $1 ~ "zip") print $1 }' info-mime-size-db.txt
gawk -F '|' '{AA=index($1,"png");if (AA) print $1 }' info-mime-size-db.txt

Simple walk through folders

This will copy files from one destination to another, based on the name or the file extension, it doesn't use or do checks for any other information about files as e.g. a mime-type or a pre-made file with the descriptions. You can modify the script depends on what kind of files you will need. This script is slow because is must go through each folder and search for files. You can download the example script «search-folder-by-folder.sh» from the SourceForge.

Config Alsa Note

ALSA advanced settings linuxSoundALSA

Make install - preparation

automoc4 (req. for kde lang compile)

libtoolize --force
aclocal
autoheader
automake --force-missing --add-missing
autoconf
./configure
make
# make install

One more make example for installation 

 export LIBS=-lXext
 ./configure --prefix=/usr --x-libraries=/usr/lib
#  make 
 make prefix="$pkgdir/usr" "libexecdir=$pkgdir/usr/bin" install

# make all-recursive

See also 

xmkmf
imake
cmake

List installed from custom or official repositories

RepoName="custom"

List all installed that are not in custom repository 

$ pacman -Ss | grep -i 'installed'  | grep '/' | grep -v -e ^"$RepoName" -e ^' ' | awk -F'[// ]' '{print $2}'

List all that are in custom repository 

$ pacman -Ss | grep -i 'installed'  | grep '/' | grep "$RepoName" | grep -v ^' ' | awk -F'[/\/ ]' '{print $2}'

Prepare a safe restoration/reset of the whole storage device

Here will be described basic theoretical steps about how to make more easier restore of the default operation system (e.g. Arch Linux), just by using "Reset" function made by you that will be very useful for beginners/common users or if you will have plans to sell computers with a preferred Linux.

  • The initial factory set up must be stored on the write protected storage device such as e.g. Secure Digital SD cards that can be physically locked into the read only mode.
  • The latest updated factory set up must be stored on the writeable storage device or on a separate partition but with limited access such as write protected mount predefined in fstab and/or use in additional mount scripts mount.* with checks which device/partition is mounted and allow mount only in READ ONLY mode for a normal user.
  • It also good to have those destinations hidden in a file managers menu.
  • The BIOS will need to be configured to boot from SD Card. The SD card need to be permanent attached and sealed(glued).

You can make your own custom Live CD with Archiso that must have functions:

  • Health monitoring tools such as smartmontools to show errors and instructions what to do if destination storage device is damaged.
  • Restore whole storage device with tools as fsarchiver from the back up image if the main partition is damaged or have it as an option
  • Retrieve updates from the internet if it is available.
  • Use latest created "ISO" if it is available to install updates

The additional ISO on a separate partition can be created after a certain amount of updates predefined by you or manually by user.

On the first login user must get an opportunity to choose a cloud server where was stored the list of all installed applications and updated configuration files.

On PC the restoration media can be stored inside the box by connecting to USB card with adaptor. Laptops are missing the ability of storing extra storage devices inside that can be used for repair purposes, but some of them can have place for the addition storage that can be connected to them such as Secure Digital SD cards where can be stored only initial "factory" ISO and optionally also the internal storage device back up image.

Virtual Box

The information about path to harddisks and the snapshots is stored between <HardDisks> .... </HardDisks> tags in the file with the .vbox extension. You can edit them manually or use this script where you will need only change the path or use defaults, assumed that .vbox is in the same directory. It will print out new configuration to stdout. 

#!/bin/bash
NewPath="${PWD}/"
Snapshots="${NewPath}/"
Filename="$1"

awk -v SetPath="$NewPath" -v SnapPath="$Snapshots" '{if(index($0,"vdi") != 0){A=$3;split(A,B,"=");
L=B[2];
 gsub(/\"/,"",L);
  sub(/^.*\//,"",L);
  sub(/^.*\\/,"",L);
 if(index($3,"{") != 0){SnapS=SnapPath}else{SnapS=""};
  print $1" "$2" location=\""SetPath SnapS L"\" "$4" "$5}
else print $0}' $Filename
Note: If you will move/clone to virtualbox in Windows then in the path name end you should use backslash \ instead of / .

Thanks

To Trible for showing some of advanced awk functionality, find duplicate x/y in a text file. 

awk -F '|' '// { Count[$3 "|" $5]++; } END { for (i in Count) { printf "%s|%s\n", i, Count[i]; }}' /path/to/file

To gregm for help about how to count a duplicate strings in an array due population of it in a python script.

To dugan about how to search integer duplicates in an array and information about using of a default fill in array without actually predefining it with a data.

from collections import defaultdict
DD = defaultdict(int)

To Alad for info about a bash spell check. I really needed it.

To 蔡依林 for the unbelievable Great voice and always the best performance ever!

Crash > test > Ouch > solution > if empty > Wiki

Retrieved from "https://wiki.archlinux.org/index.php?title=User:Andy_Crowd&oldid=782395"