2021年01月27日 に更新

オレオレLINUX便利コマンド

スタック・オーバーフローはプログラマーとプログラミングに熱心な人のためのQ&Aサイトです。すぐ登録できます。 登録してこのコミュニティに参加する どなたでも質問していただけます どなたでも回答していただけます 最もよい回答には賛成票が入り、最上位に上がります コマンドラインで読み込んだファイルに上書きするには 質問する 質問日 3 年、8 か月前 アクティブ 2 年、1 か月前 閲覧数 5,970件 7 $ cat hoge.txt > hoge.txt これを実行すると、「hoge.txtの中身がそのままhoge.txtに書き出されて結果的に中身は同じになる」と思っていたのですが、実行してみるとファイルの中身が空になり、容量も0バイトになってしまいます。 他のコマンドでも同様に読み込みファイルにたいして>をつかって上書きしようとすると中身が消えてしまいます。 上書きしたい場合はどうしたらよいのでしょうか。 linux unix 共有 この質問を改善する フォロー 質問日時: 17å¹´5月25日 13:21 snxpwfsnxpwf 2,03022 個の金バッジ2525 個の銀バッジ5050 個の銅バッジ 2 このコマンドを実行する目的は何でしょうか?(知的好奇心という意味では興味ありますが)。上書きとは言いますが内容の変更を伴わないのであれば、例えばtouch hoge.txtでタイムスタンプだけ更新する方法もあります。 – cubick♦ 17å¹´5月25日 16:54 コメント追加  |  4 件の回答 4 アクティブ 古い順 票 9 シェルがコマンドラインを解釈した後、hoge.txt をオープンして 0 バイトにしてから cat を起動するので当然の結果です。実行したいコマンドラインの前に hoge.txt を一時ファイルにコピーして利用するか、実行結果を一時ファイルに出力したあと上書きする必要があります。 moreutils の sponge(1) が利用できる環境ならば、一時ファイルは不要で次のように実行できます。 cat hoge.txt |sponge hoge.txt なお、ほかの回答にある元ファイルを一旦削除する方法は、元ファイルのファイル所有者とアクセス権が失なわれる可能性があるので注意が必要です。 共有 この回答を改善する フォロー 編集日時: 18å¹´12月6日 3:46 回答日時: 17å¹´5月25日 15:46 fumiyasfumiyas 2,76188 個の銀バッジ1919 個の銅バッジ コメント追加  |  5 黒魔術的ではありますが、以下のようにするとUNIXとシェルの機能のみで中間ファイルを利用することなく実現できます。 (rm -f hoge.txt && cat > hoge.txt) < hoge.txt cat だと分かりにくいので、sort を使った例: % cat hoge.txt 3 1 2 % (rm -f hoge.txt && sort > hoge.txt) < hoge.txt % cat hoge.txt 1 2 3 オープンしたファイルを削除しても、クローズされるまではファイル本体が残っている (読み書き可能) であることを利用 (悪用?) しています。 参考: https://twitter.com/kuwashima/status/854716064109699072 共有 この回答を改善する フォロー 回答日時: 17å¹´5月26日 1:18 yasuyasu 4,39333 個の金バッジ1616 個の銀バッジ2525 個の銅バッジ コメント追加  |  1 リダイレクトはコマンドが実行される前に shell が処理してしまってますから、コマンドが実行された時には既にファイルは空になっています。 出力を一旦別ファイルに格納するしかないのでは? cat hoge.txt > tmp mv tmp hoge.txt 共有 この回答を改善する フォロー 回答日時: 17å¹´5月25日 13:58 masmmasm 3,16222 個の金バッジ1010 個の銀バッジ1717 個の銅バッジ コメント追加  |  0 GNU cat だとその構文はエラーになります。 $ cat hoge.txt > hoge.txt cat: hoge.txt: input file is output file 次のようなやり方はどうでしょう。 $ cat hoge.txt | bash -c 'cat - > hoge.txt' 共有 この回答を改善する フォロー 回答日時: 17å¹´5月25日 14:24 take88take88 7,58411 個の金バッジ1212 個の銀バッジ2929 個の銅バッジ 4 運がよければうまくいく場合もあるでしょうが、駄目な場合もあります。恐らく hoge.txt のサイズが大きいほど失敗しやすくなるでしょうね。 – fumiyas 17å¹´5月25日 15:55 4 基本的にリダイレクト先のことをプロセスは知らないのでGNUのcatが検出できるのが不思議だったのですが、ファイルディスクリプタから調べたfstatのdeviceやらi-nodeやらを見てがんばっているみたいですね。試してみた限りでは、ext3,4 では出来ましたが、btrfs,xfs では駄目でした。 git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/… – sakuro 17å¹´5月26日 9:58 コメント追加  |  求めていた回答ではありませんか? linux unix のタグが付いた他の質問を参照するか、自分で質問をする。 メタでのおすすめ Opt-in alpha test for a new Stacks editor ヘルプセンター翻訳:レビューキュー記事を日本語に訳しましょう! モデレーションレポート (2020å¹´) 関連する質問 11 grepコマンドでのファイル指定について 2 grepとxargsを組み合わせて複数文字列の検索を行いたい 2 圧縮ファイルの展開をundoしたい 2 `/bin/bash ./hoge.sh > out1.txt 2>&1`では、`bash -x`の結果がファイルに出力されませんでした。なぜでしょうか? 1 Debian 10でbashを起動したときに、bash: dircolors: command not foundと出ます。 3 Linuxで7文字以下の行をファイルから削除したい 1 scriptコマンドで生成したログファイルを文字化けせずに開きたい 5 Linuxで誤って上書きしてしまったファイルを復元したい 0 カレント以下のファイルの中身をまとめて見る方法 質問のフィード RSSの購読 質問のフィード このRSSフィードを購読するには、このURLをコピーしてRSSリーダーに貼り付けてください。 スタック・オーバーフロー をより快適に使うためには JavaScript を有効化してください
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up. Sign up to join this community Anybody can ask a question Anybody can answer The best answers are voted up and rise to the top How to output only file names (with spaces) in ls -Al? Ask Question Asked 7 years, 10 months ago Active 10 months ago Viewed 275k times 108 27 I should echo only names of files or directories with this construction: ls -Al | while read string do ... done ls -Al output : drwxr-xr-x 12 s162103 studs 12 march 28 12:49 personal domain drwxr-xr-x 2 s162103 studs 3 march 28 22:32 public_html drwxr-xr-x 7 s162103 studs 8 march 28 13:59 WebApplication1 For example if I try: ls -Al | while read string do echo "$string" | awk '{print $9} done then output only files and directories without spaces. If file or directory have spaces like "personal domain" it will be only word "personal". I need very simple solution. Maybe there is better solution than awk. linux command-line ls Share Improve this question Follow edited Mar 4 '15 at 18:03 kenorb 15.2k77 gold badges110110 silver badges145145 bronze badges asked Mar 30 '13 at 14:59 Alex ZernAlex Zern 1,27933 gold badges1111 silver badges88 bronze badges 3 Why not just ls -Al *' '*? Parsing ls's output never leads to anything good. – manatwork Mar 30 '13 at 15:01 1 If you need something that will work in any *nix try avoiding using ls -Al | while. A simple and more reliable way is for string in *; do echo "$string"; done. – forcefsck Mar 30 '13 at 15:17 need only with this construction..okay,not simple,any solution! – Alex Zern Mar 30 '13 at 15:31 2 Why do you need ls? Parsing ls and "work in all *nix and will not break if something happened" do not go well together. – terdon♦ Mar 30 '13 at 15:59 2 mywiki.wooledge.org/ParsingLs – Braiam May 10 '14 at 23:29  |  show 2 more comments 8 Answers 8 Active Oldest Votes 131 You really should not parse the output of ls. If this is a homework assignment and you are required to, your professor does not know what they're talking about. Why don't you do something like this: The good... find ./ -printf "%f\n" or for n in *; do printf '%s\n' "$n"; done ...the bad... If you really really want to use ls, you can make it a little bit more robust by doing something like this: ls -lA | awk -F':[0-9]* ' '/:/{print $2}' ...and the ugly If you insist on doing it the wrong, dangerous way and just have to use a while loop, do this: ls -Al | while IFS= read -r string; do echo "$string" | awk -F':[0-9]* ' '/:/{print $2}'; done Seriously though, just don't. Share Improve this answer Follow edited Mar 23 '18 at 10:37 answered Mar 30 '13 at 16:11 terdon♦terdon 185k4949 gold badges355355 silver badges545545 bronze badges drwx------ 2 s162103 studs 3 oct. 9 2012 .ssh drwxr-xr-x 3 s162103 studs 6 oct. 25 09:02 .subversion drwxrwxrwt 3 s162103 studs 3 nov. 15 2012 .TempoItems sometimes i've got date without time, only year and gawk return empty – Alex Zern Mar 30 '13 at 16:30 1 @AlexZern Well, that's one of the reasons you don't parse the output of ls. – terdon♦ Mar 30 '13 at 16:33 2 @AlexZern why do you need the -l switch anyway? If all you want is the file names, just run ls -A. Using -l just makes your life harder since you now have to parse the output. – terdon♦ Mar 30 '13 at 16:37 because I need additional information like access, date, owner,etc. And I wrote a huge script ,which works well, except print file and dir names with spaces. – Alex Zern Mar 30 '13 at 16:46 3 OK then. What we have here is an XY problem. ls is just not the best way of getting what you need. Why don't you post a new question explaining the information you want to collect and we can suggest ways of doing it. Alternatively, you can use your script as is, and use one of the suggestions above to print the name only. But seriously, don't parse ls, just tell us exactly what you are trying to do in a new question. – terdon♦ Mar 30 '13 at 16:49  |  show 3 more comments 113 Is there some reason that ls -A1* won't work? E.g.: $ touch file1 file2 file\ with\ spaces $ ls -Al total 0 -rw-r--r-- 1 bahamat bahamat 0 Mar 30 22:31 file1 -rw-r--r-- 1 bahamat bahamat 0 Mar 30 22:31 file2 -rw-r--r-- 1 bahamat bahamat 0 Mar 30 22:31 file with spaces $ ls -A1 file1 file2 file with spaces $ * Note: that's a capital letter A and the number one. Share Improve this answer Follow answered Mar 31 '13 at 5:35 bahamatbahamat 31.5k33 gold badges6161 silver badges9696 bronze badges 2 Cleanest solution. Thank you – Luciano Fantuzzi Feb 26 '20 at 22:34 Works perfectly. – Farhan A. Rafi Aug 13 '20 at 8:31 add a comment  |  69 I wonder why no one mentioned this simple command: ls -a | sort Share Improve this answer Follow answered Aug 11 '16 at 8:14 BenBen 79255 silver badges66 bronze badges 2 Brilliant! Any idea how piping to sort causes multi-column block to be unpivoted into single column? – msciwoj Sep 14 '16 at 10:49 @msciwoj it just takes any strings separated by whitespaces and prints them sorted separated by endlines – Ben Sep 14 '16 at 12:32 5 It works here because ls reverts to ls -1 (single columnd output) when its output doesn't go to a terminal (and disables non-printable character quoting/replacement; @Kusalananda, your ls is not POSIX compliant if it still does). ls output is already sorted. So piping it to sort will at best do nothing, and at worse mangle the output (if file names contain newline characters) or fail for filenames that are not text. ls -a | cat would be better. Or even better ls -A1 as already mentioned (and because you generally do want the replacements/quoting when displayed on a terminal) – Stéphane Chazelas Mar 23 '18 at 11:22 1 @StéphaneChazelas Noted. ls behaves differently when output is not a terminal. – Kusalananda♦ Mar 23 '18 at 11:25 add a comment  |  5 You cannot parse the output of ls, let alone ls -l because newline, just like space is as valid a character as any in a filename. Also, you'll need to consider symlinks that have an output like ... foo -> bar. Why would you use -l anyway if you only want the file name? Just do: for file in *; do ... done If you want to include dot files (except . and ..), depending on the shell: zsh: for file in *(ND); do ... done bash: shopt -s nullglob dotglob for file in *; do ... done ksh93: FIGNORE='@(.|..)' for file in ~(N)*; do ... done POSIXly: for file in .[!.]* ..?* *; do [ -e "$file" ] || [ -L "$file" ] || continue ... done Share Improve this answer Follow edited Mar 23 '18 at 12:08 answered Apr 10 '15 at 19:24 Stéphane ChazelasStéphane Chazelas 406k7171 gold badges796796 silver badges12101210 bronze badges add a comment  |  2 ls -Al | tr -s ' ' | cut -f9- -d' ' compress multiple spaces into single spaces with tr then you can use cut to split on the fields Share Improve this answer Follow answered Mar 4 '15 at 21:16 simpleusersimpleuser 32633 silver badges1717 bronze badges In my case where I knew the full path just not the file name, cut worked best. Full command for getting the most recent with version-aware sorting: ls /workdir/node-* | cut -f 3 -d / | sort -V -r | head -n 1 – Samuel Neff Feb 5 '20 at 20:23 add a comment  |  2 Try (bash, zsh, ksh93 syntax): find . -type f -print0 | while IFS= read -r -d '' filename; do ... done This goes recursive and lists only on normal files (i.e. no dirs or symlinks/fifos/devices...). See also: List only regular files (but not directories) in current directory Why you shouldn't parse the output of ls(1) at wooledge wiki Why not parse ls? at unix SE Share Improve this answer Follow edited Apr 13 '17 at 12:37 Community♦ 1 answered Mar 4 '15 at 18:01 kenorbkenorb 15.2k77 gold badges110110 silver badges145145 bronze badges add a comment  |  2 Probably the easiest way to print all file names in directory is just: echo * Downside - won't print hidden files. Share Improve this answer Follow answered Mar 17 '20 at 14:01 GrzegorzGrzegorz 2111 bronze badge Depending on the shell, there may be shell options to set to make the pattern expand to hidden names (e.g. dotglob in the bash shell), or special patterns that does so (like *(D) in the zsh shell). This is mentioned in another answer. – Kusalananda♦ Mar 17 '20 at 14:16 add a comment  |  1 How about: for file in * .[!.]* do printf "%s\n" "$file" done Share Improve this answer Follow answered Mar 30 '13 at 16:13 ScrutinizerScrutinizer 1,09455 silver badges66 bronze badges add a comment  |  Not the answer you're looking for? Browse other questions tagged linux command-line ls or ask your own question. The Overflow Blog A deeper dive into our May 2019 security incident Podcast 307: Owning the code, from integration to delivery Featured on Meta Opt-in alpha test for a new Stacks editor Linked 4 List only regular files 0 Must parse ls -Al output and get file or directory name 232 Why *not* parse `ls` (and what to do instead)? 142 List only regular files (but not directories) in current directory 2 How to use find command to list file names but not path? 0 How to retrieve folders name in a directory with bash Related 11 Find directories that do not contain subdirectories 8 Using find to list all files under certain directory 32 Simple way to see the content of directories in Linux/UNIX file systems 218 Why is 'ls' suddenly wrapping items with spaces in single quotes? 1 For a large directory, create a variable of the filenames which include lines which include the text string stored in another variable 5 Print two files in two columns 1 delete files by iterating an array? 3 Struggling with output of grep into an array 0 Linux Internal working of `cat` Hot Network Questions How did US Supreme court Justice John Roberts "let it be known" that he did not wish to preside over the 2nd presidential impeachment? What is an effective way to evaluate and assess employees on a non-management career track? Why are they so good at magic? How Can Vampires Obtain Donors? Are all atoms spherically symmetric? If so, why are atoms with half-filled/filled sub-shells often quoted as 'especially' spherically symmetric? Why does `xstring`s `\StrBefore` work, but not `\IfBeginWith`? How can I motivate the teaching assistants to grade more strictly? What type of lightbulb is this? OpenStreetMap baselayer of just one country Biking Target Speeds Are there any diacritics not on the top or bottom of a letter? How far can a wolf drag a 175 lb human on tundra? Is a license recommended for a private repository or is it pointless? Is the Wi-Fi in high-speed trains in China reliable and fast enough for audio or video conferences? Could double jeopardy protect a murderer who bribed the judge and jury to be declared not guilty? Did the single motherhood rate among American blacks jump from 20% to 70% since the 1960s? Order of operations and rounding for microcontrollers Why can this derived class be constructed with `{}` and not with `()` on C++17? Inserting © (copyright symbol) using Microsoft Word Is it offensive to kill my gay character at the end of my book? Did Barry Goldwater claim peanut butter is good shaving cream? How do you say "I knew I should have gone"? What do "tangential and centripetal acceleration" mean for non-circular motion? Save performance with large update on Index with Included Column more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. Unix & Linux Stack Exchange works best with JavaScript enabled
ホーム / Linux入門 / Linuxコマンド / sortコマンドについて詳しくまとめました 【Linuxコマンド集】 sortはファイルの内容を並べ替えるコマンドだ。 目次1 sortコマンドの基本1.1 コマンドの基本動作2 sortコマンドの結果をファイルに出力する3 sortコマンドのオプションたち3.1 オプションの一覧3.2 -nオプション(--numeric-sort):数値として並べ替える3.3 -rオプション(--reverse):降順で並べ替える3.4 -tオプション(--field-separator):項目の区切りを指定する3.5 -kオプション(--key):キーを指定して並べ替える3.6 -uオプション(--unique-r):重複行を省いて並べ替える3.7 -fオプション(--ignore-case-u):大文字小文字を関係なく並べ替える4 sort関連コマンド4.1 cutコマンド sortコマンドの基本 コマンドの基本動作 使い方は、次のとおりだ。 $ sort ファイル名 ファイルdata1.txtは、次のような内容だ。 10 5 32 1 1234 105321 ファイルdata1.txtを並べ替えする場合は、次のコマンドだ。 $ sort data1.txt データが昇順で並べ替えされた。 並べ替えの内容が、1、10、32、5という順序になるのは、1文字目の小さい順に並ぶからだ。 sortコマンドの標準では文字として左側から昇順に並べ替えを行った並び方になる。 次に説明するが、オプションを使用することで、文字としてではなく数値として並べ替えたり、降順で並べ替えたりすることができる。 sortコマンドの結果をファイルに出力する sortコマンドは、ディスプレイに結果を表示するが、大量のデータを処理した場合は、ファイルに出力した方が都合がよい。 ファイルに出力するには、リダイレクトする。 書式は次の通りだ。 $ sort 並べ替え対象ファイル > 並べ替え出力ファイル data1.txtの並べ替えの結果をファイルdata1_sort.txtとして保存する場合は次の通りだ。 $ sort data1.txt > data1_sort.txt sortコマンドのオプションたち オプションの一覧 後から詳細をご紹介するが、まずは一覧で見てみよう。 オプション-n(オプション--numeric-sort) 数値として並べ替える。 オプション-r(オプション--reverse) 降順で並べ替える。 オプション-tオプション--field-separator) 項目の区切りを指定する。 オプション-k(オプション--key) キーを指定して並べ替える。 オプション-u(オプション--unique-r) 重複行を省いて並べ替える。 オプション-f(オプション--ignore-case-u) 大文字小文字を関係なく並べ替える。 一つずつ見ていこう。 -nオプション(--numeric-sort):数値として並べ替える 数値として並べ替えを行う書式は次のとおりだ。 $ sort -n ファイル名 ファイルdata1.txtを数値として並べ替えする場合は、次のコマンドだ。 $ sort -n data1.txt ファイルdata1.txtが数値として並べ替えされた。 -rオプション(--reverse):降順で並べ替える 数値として並べ替えを行う書式は次のとおりだ。 $ sort -r ファイル名 ファイルdata1.txtを数値として降順で並べ替えする場合は、次のコマンドだ。 $ sort -r -n data1.txt ファイルdata1.txtを数値として降順で並べ替えされた。 -tオプション(--field-separator):項目の区切りを指定する ファイルdata2.txtは、次のような内容だ。 01:orange:250:Tokyo 02:apple:230:Nagoya 03:apple:130:Nagoya 04:grape:450:Tokyo 05:orange:150:Osaka 12345 01:orange:250:Tokyo02:apple:230:Nagoya03:apple:130:Nagoya04:grape:450:Tokyo05:orange:150:Osaka 「:」で区切られている。 さまざまなデータファイルは、このようにある文字で区切られていることが多い。 このような区切り文字を指定する場合に-tオプションを使用する。 書式は次の通りだ。 $ sort -t 区切り文字 ファイル名 -tオプションは、次の-kオプションと組み合わせて利用する。 -kオプション(--key):キーを指定して並べ替える 並べ替えの基準の項目を指定して並べ替えを行う書式は次のとおりだ。 $ sort -k 並べ替えに使う項目番号 -t 区切り文字 ファイル名 ファイルdata2.txtを「:」で区切り、2項目目の品名で並べ替えする場合は、次のコマンドだ。 $ sort -k 2 -t ":" data2.txt ファイルdata2.txtが2項目目の品名でアルファベット順に並べ替えされた。 ただし、上記の場合だと2項目目以降で並べ替えとなる。 2項目目だけで並べ替えを行う場合は、次のコマンドだ。 $ sort -k 2,2 -t ":" data2.txt 2項目目だけで並べ替えを行った。 「,」を挟むことで優先項目の開始と優先項目の終了を指定できる。 2項目目と3項目目で並べ替えを行う場合は、次のコマンドだ。 $ sort -k 2,3 -t ":" data2.txt この場合、2項目目を最優先、3項目目がその次に優先となる。 $ sort -k 2,4 -t ":" data2.txt この場合、2項目目を最優先、4項目目がその次に優先となるのではなく、2項目目が第一優先、3項目目が第二優先、4項目目が第三優先となる。 -uオプション(--unique-r):重複行を省いて並べ替える ファイルdata3.txtは、次のような内容だ。 Taro Jim Kenta Taro Hanako Kenta 123456 TaroJimKentaTaroHanakoKenta TaroとKentaが重複している。 この重複したデータを削除するオプションが-uオプションで、書式は次のとおりだ。 $ sort -u ファイル名 ファイルdata3.txtの中から重複データを削除し、並べ替えする場合は、次のコマンドだ。 $ sort -u data3.txt ファイルdata3.txtが重複データを削除しつつ、アルファベット順で並べ替えされた。 -fオプション(--ignore-case-u):大文字小文字を関係なく並べ替える ファイルdata4.txtは、次のような内容だ。 Tokyo Osaka Nagoya OSAKA tokyo 12345 TokyoOsakaNagoyaOSAKAtokyo Tokyoとtokyo、OsakaとOSAKAのように大文字小文字が混在している。 sortオプションの標準では大文字と小文字は区別するために、aからzの次にAからZが並ぶ。 この大文字小文字を区別しないオプションが-fオプションで、書式は次のとおりだ。 $ sort -f ファイル名 ファイルdata4.txtを大文字小文字を区別せずに並べ替えする場合は、次のコマンドだ。 $ sort -f data4.txt ファイルdata4.txtを大文字小文字を区別せずに並べ替えされた。 sort関連コマンド 最後にsortコマンドに関連して、基本的なテキスト処理のコマンドも紹介しておく。 cutコマンド 文字列を分離する。 まとめ 今回は、sortコマンドを使ったテキスト形式のファイルの並べ替えの方法を解説した。 様々なデータはcsv形式と呼ばれるカンマ区切りのテキスト形式のファイルで管理されることが多い。 また、並べ替えた結果は、リダイレクトすることでファイルに書き出せることも覚えておこう。 短期間でエンジニアになる方法 ・「まったくの初心者だけどエンジニアになりたい!」・「プログラマーとして転職をしたい!」という方はリナックスアカデミーの資料を見てみてください。短期間で未経験からエンジニアになることができるスクールとして15年間選ばれ続けてきた理由やノウハウが載った資料です。エンジニアの入り口に立つために必要な勉強や技術の最新動向、本当に使えるIT資格、学習に役立つ国からの奨励金などの情報が詰まっています。無料で2,3日中にお手元にお届けします。 資料を見てみる SNSでもご購読できます。 Twitterでフォローする Feedlyでフォローする リナックスアカデミー 【ITエンジニア養成スクール & IT研修専門企業のリナックスアカデミーです。】エンジニアの入り口に立つために役立つようなコンテンツを日々ご提供していきます。講師や代表やスタッフ陣が毎日楽しく書いています。ご質問・ご指摘等はぜひコメントください。 コメント shinriyo より: 2018年6月7日 6:57 PM -kのオプションは各順番は影響しませんよね? 左から強い気がします。 例えば3番目でソートした後で2番目でソートは sort -k 3,2 -t ":" data2.txt にしてもだめでした。 返信 リナックスアカデミー より: 2018年9月17日 4:18 PM ご愛読いただきありがとうございます。 -kオプションは複数記述が可能ですので、ソートの順番を右→左のように行う場合には以下のようになります。 sort -k 3,3 -k 2,2 -t ":" data2.txt 返信 コメントを残す コメントをキャンセル リナックスアカデミー RANKING ファイルやディレクトリをコピーするcpコマンドの使い方!【Linuxコマンド集】ファイルなどを検索する!findコマンドの詳細まとめ【Linuxコマンド集】grepコマンドの詳細まとめました【Linuxコマンド集】【Windows編】フリーテキストエディタ13選!初心者にもおすすめ【初心者向け】3分でわかるLinuxでできることまとめ 10選【目的別】初心者におすすめのLinuxディストリビューション7選【初心者必見】コマンドプロンプトの使い方と覚えておきたい便利な機能lsコマンドの使い方と覚えたい15のオプション【Linuxコマンド集】【今更聞けない!】32bitと64bitの違いとは? まとめファイル・ディレクトリを削除するrmコマンドについて詳しく!【Linuxコマンド集】 最近の投稿 【初心者向け】Pythonの特徴8選 【1分で読める】プログラミング講師になる7つのメリット 【PHP入門者向け】 PHPの文字列リテラルについて知ろう 【PHP入門者向け】 文字列の中の変数展開についてまとめました 【ざっくり理解する】 IPv6アドレスとは? 【初心者向け】システムエンジニアとは?その種類と役割は? 【Java & Tomcat】HttpServletResponseインターフェイスについて知ろう 【Java & Tomcat】Webアプリケーションの設定(web.xmlの設定)※サンプルプログラム付き 【Java & Tomcat】Webアプリケーションの設定(マッピング)※サンプルプログラム付き 【Java & Tomcat】コンテンツタイプの設定(setContentType)※サンプルプログラム付き 【Java & Tomcat】サーブレットのライフサイクル ※サンプルプログラム付き 【Java & Tomcat】文字出力用のストリームの取得(getWriter)※サンプルプログラム付き 【Java & Tomcat】リダイレクトの設定(sendRedirect)※サンプルプログラム付き 【Java & Tomcat】サーブレットからフォームデータを取得する(サンプルコード付き) 【Java & Tomcat】サーブレット開発をEclipseでするための初期設定を画像付きで分かりやすく解説 各種ページ 会社概要 最新記事一覧 無料資料請求|最短でエンジニアになるための情報が満載 リンク情報ITエンジニアスクール IT研修サービス 短期IT研修サービス 株式会社リカレント プログラミング講師募集 ゼロからエンジニアを目指す方へ短期間でエンジニアになる方法 必要な勉強や技術の最新動向、本当に使えるIT資格、学習に役立つ国からの奨励金などの情報を無料でお届け 資料を見てみる
Join Stack Overflow to learn, share knowledge, and build your career. Sign up with email Sign up Sign up with Google Sign up with GitHub Sign up with Facebook Teams Q&A for Work Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information. Learn more Bash tool to get nth line from a file Ask Question Asked 9 years, 8 months ago Active 3 months ago Viewed 518k times 670 212 Is there a "canonical" way of doing that? I've been using head -n | tail -1 which does the trick, but I've been wondering if there's a Bash tool that specifically extracts a line (or a range of lines) from a file. By "canonical" I mean a program whose main function is doing that. bash shell unix awk sed Share Improve this question Follow edited Jun 4 '17 at 22:12 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges asked May 16 '11 at 19:33 Vlad VivdovitchVlad Vivdovitch 7,44766 gold badges1717 silver badges2020 bronze badges 10 The "Unix way" is to chain tools that do their respective job well. So I think you already found a very suitable method. Other methods include awk and sed and I'm sure someone can come up with a Perl one-liner or so as well ;) – 0xC0000022L May 16 '11 at 19:35 3 The double-command suggests that the head | tail solution is sub-optimal. Other more nearly optimal solutions have been suggested. – Jonathan Leffler May 16 '11 at 19:57 Have you ran any benchmarks on which solution is the fastest for an average case? – Marcin May 17 '11 at 11:56 6 Benchmarks (for a range) at cat line X to line Y on a huge file on Unix & Linux. (cc @Marcin, in case you're still wondering after two+ years) – Kevin Aug 8 '13 at 14:13 9 The head | tail solution does not work, if you query a line that does not exist in the input: it will print the last line. – jarno Mar 1 '16 at 0:24  |  show 1 more comment 19 Answers 19 Active Oldest Votes 885 head and pipe with tail will be slow for a huge file. I would suggest sed like this: sed 'NUMq;d' file Where NUM is the number of the line you want to print; so, for example, sed '10q;d' file to print the 10th line of file. Explanation: NUMq will quit immediately when the line number is NUM. d will delete the line instead of printing it; this is inhibited on the last line because the q causes the rest of the script to be skipped when quitting. If you have NUM in a variable, you will want to use double quotes instead of single: sed "${NUM}q;d" file Share Improve this answer Follow edited Feb 16 '16 at 4:22 tripleee 133k2323 gold badges195195 silver badges257257 bronze badges answered May 16 '11 at 19:38 anubhavaanubhava 639k5555 gold badges446446 silver badges527527 bronze badges 49 For those wondering, this solution seems about 6 to 9 times faster than the sed -n 'NUMp' and sed 'NUM!d' solutions proposed below. – Skippy le Grand Gourou Feb 18 '14 at 16:48 76 I think tail -n+NUM file | head -n1 is likely to be just as fast or faster. At least, it was (significantly) faster on my system when I tried it with NUM being 250000 on a file with half a million lines. YMMV, but I don't really see why it would. – rici Mar 25 '14 at 2:43 @rici and you can easily choose how many lines past that point by changing head -n1 to head -nNUM2, you should make this it's own answer – Hashbrown Sep 1 '20 at 2:22 How do you use this with a range of lines from line n to m? – Herman Toothrot Oct 2 '20 at 10:16 For range better use sed -n '2,5{p;5q;}' file – anubhava Oct 2 '20 at 10:18  |  show 2 more comments 335 sed -n '2p' < file.txt will print 2nd line sed -n '2011p' < file.txt 2011th line sed -n '10,33p' < file.txt line 10 up to line 33 sed -n '1p;3p' < file.txt 1st and 3th line and so on... For adding lines with sed, you can check this: sed: insert a line in a certain position Share Improve this answer Follow edited May 23 '17 at 10:31 Community♦ 111 silver badge answered May 16 '11 at 19:39 jm666jm666 52.2k1414 gold badges8888 silver badges157157 bronze badges 2 Why is the '<' necessary in this case? Wouldn't I achieve the same output without it? – Rafael Barbosa May 27 '13 at 14:10 7 @RafaelBarbosa the < in this case is not necessary. Simply, it is my preference using redirects, because me often used redirects like sed -n '100p' < <(some_command) - so, universal syntax :). It is NOT less effective, because redirection are done with shell when forking itself, so... it is only a preference... (and yes, it is one character longer) :) – jm666 May 27 '13 at 16:11 1 @jm666 Actually it's 2 characters longer since you would normally put the '<' as well as an extra space ' ' after < as oppposed to just one space if you hadn't used the < :) – rasen58 Aug 11 '16 at 20:59 2 @rasen58 the space is an character too? :) /okay, just kidding - youre right/ :) – jm666 Aug 12 '16 at 6:58 1 This is about 5 times slower than the tail / head combination when reading a file with 50M rows – duhaime Jun 4 '18 at 14:42  |  show 3 more comments 104 I have a unique situation where I can benchmark the solutions proposed on this page, and so I'm writing this answer as a consolidation of the proposed solutions with included run times for each. Set Up I have a 3.261 gigabyte ASCII text data file with one key-value pair per row. The file contains 3,339,550,320 rows in total and defies opening in any editor I have tried, including my go-to Vim. I need to subset this file in order to investigate some of the values that I've discovered only start around row ~500,000,000. Because the file has so many rows: I need to extract only a subset of the rows to do anything useful with the data. Reading through every row leading up to the values I care about is going to take a long time. If the solution reads past the rows I care about and continues reading the rest of the file it will waste time reading almost 3 billion irrelevant rows and take 6x longer than necessary. My best-case-scenario is a solution that extracts only a single line from the file without reading any of the other rows in the file, but I can't think of how I would accomplish this in Bash. For the purposes of my sanity I'm not going to be trying to read the full 500,000,000 lines I'd need for my own problem. Instead I'll be trying to extract row 50,000,000 out of 3,339,550,320 (which means reading the full file will take 60x longer than necessary). I will be using the time built-in to benchmark each command. Baseline First let's see how the head tail solution: $ time head -50000000 myfile.ascii | tail -1 pgm_icnt = 0 real 1m15.321s The baseline for row 50 million is 00:01:15.321, if I'd gone straight for row 500 million it'd probably be ~12.5 minutes. cut I'm dubious of this one, but it's worth a shot: $ time cut -f50000000 -d$'\n' myfile.ascii pgm_icnt = 0 real 5m12.156s This one took 00:05:12.156 to run, which is much slower than the baseline! I'm not sure whether it read through the entire file or just up to line 50 million before stopping, but regardless this doesn't seem like a viable solution to the problem. AWK I only ran the solution with the exit because I wasn't going to wait for the full file to run: $ time awk 'NR == 50000000 {print; exit}' myfile.ascii pgm_icnt = 0 real 1m16.583s This code ran in 00:01:16.583, which is only ~1 second slower, but still not an improvement on the baseline. At this rate if the exit command had been excluded it would have probably taken around ~76 minutes to read the entire file! Perl I ran the existing Perl solution as well: $ time perl -wnl -e '$.== 50000000 && print && exit;' myfile.ascii pgm_icnt = 0 real 1m13.146s This code ran in 00:01:13.146, which is ~2 seconds faster than the baseline. If I'd run it on the full 500,000,000 it would probably take ~12 minutes. sed The top answer on the board, here's my result: $ time sed "50000000q;d" myfile.ascii pgm_icnt = 0 real 1m12.705s This code ran in 00:01:12.705, which is 3 seconds faster than the baseline, and ~0.4 seconds faster than Perl. If I'd run it on the full 500,000,000 rows it would have probably taken ~12 minutes. mapfile I have bash 3.1 and therefore cannot test the mapfile solution. Conclusion It looks like, for the most part, it's difficult to improve upon the head tail solution. At best the sed solution provides a ~3% increase in efficiency. (percentages calculated with the formula % = (runtime/baseline - 1) * 100) Row 50,000,000 00:01:12.705 (-00:00:02.616 = -3.47%) sed 00:01:13.146 (-00:00:02.175 = -2.89%) perl 00:01:15.321 (+00:00:00.000 = +0.00%) head|tail 00:01:16.583 (+00:00:01.262 = +1.68%) awk 00:05:12.156 (+00:03:56.835 = +314.43%) cut Row 500,000,000 00:12:07.050 (-00:00:26.160) sed 00:12:11.460 (-00:00:21.750) perl 00:12:33.210 (+00:00:00.000) head|tail 00:12:45.830 (+00:00:12.620) awk 00:52:01.560 (+00:40:31.650) cut Row 3,338,559,320 01:20:54.599 (-00:03:05.327) sed 01:21:24.045 (-00:02:25.227) perl 01:23:49.273 (+00:00:00.000) head|tail 01:25:13.548 (+00:02:35.735) awk 05:47:23.026 (+04:24:26.246) cut Share Improve this answer Follow edited Jun 4 '17 at 22:11 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Aug 30 '16 at 0:29 CaffeineConnoisseurCaffeineConnoisseur 2,93544 gold badges1515 silver badges1616 bronze badges 4 I wonder how long just cat'ting the entire file into /dev/null would take. (What if this was only a hard disk benchmark?) – sanmai Mar 23 '19 at 2:52 I feel a perverse urge to bow at your ownership of a 3+ gig text file dictionary. Whatever the rationale, this so embraces textuality :) – Stabledog Feb 4 '20 at 14:59 The overhead of running two processes with head + tail will be negligible for a single file, but starts to show when you do this on many files. – tripleee Oct 19 '20 at 11:16 add a comment  |  58 With awk it is pretty fast: awk 'NR == num_line' file When this is true, the default behaviour of awk is performed: {print $0}. Alternative versions If your file happens to be huge, you'd better exit after reading the required line. This way you save CPU time See time comparison at the end of the answer. awk 'NR == num_line {print; exit}' file If you want to give the line number from a bash variable you can use: awk 'NR == n' n=$num file awk -v n=$num 'NR == n' file # equivalent See how much time is saved by using exit, specially if the line happens to be in the first part of the file: # Let's create a 10M lines file for ((i=0; i<100000; i++)); do echo "bla bla"; done > 100Klines for ((i=0; i<100; i++)); do cat 100Klines; done > 10Mlines $ time awk 'NR == 1234567 {print}' 10Mlines bla bla real 0m1.303s user 0m1.246s sys 0m0.042s $ time awk 'NR == 1234567 {print; exit}' 10Mlines bla bla real 0m0.198s user 0m0.178s sys 0m0.013s So the difference is 0.198s vs 1.303s, around 6x times faster. Share Improve this answer Follow edited May 20 '20 at 13:14 answered Jan 22 '14 at 9:49 fedorqui 'SO stop harming'fedorqui 'SO stop harming' 221k7676 gold badges450450 silver badges505505 bronze badges This method is always going to be slower because awk attempts to do field splitting. The overhead of field splitting can be reduced by awk 'BEGIN{FS=RS}(NR == num_line) {print; exit}' file – kvantour May 15 '20 at 8:46 The real power of awk in this method comes forth when you want to concatenate line n1 of file1, n2 of file2, n3 or file3 ... awk 'FNR==n' n=10 file1 n=30 file2 n=60 file3. With GNU awk this can be sped up using awk 'FNR==n{print;nextfile}' n=10 file1 n=30 file2 n=60 file3 . – kvantour May 15 '20 at 8:51 @kvantour indeed, GNU awk's nextfile is great for such things. How come FS=RS avoids field splitting? – fedorqui 'SO stop harming' May 15 '20 at 8:52 1 FS=RS does not avoid field splitting, but it only parses the $0 ones and only assigns one field because there is no RS in $0 – kvantour May 15 '20 at 8:53 @kvantour I've been doing some tests with FS=RS and did not see difference on the timings. What about me asking a question about it so you can expand? Thanks! – fedorqui 'SO stop harming' May 20 '20 at 13:15 add a comment  |  35 According to my tests, in terms of performance and readability my recommendation is: tail -n+N | head -1 N is the line number that you want. For example, tail -n+7 input.txt | head -1 will print the 7th line of the file. tail -n+N will print everything starting from line N, and head -1 will make it stop after one line. The alternative head -N | tail -1 is perhaps slightly more readable. For example, this will print the 7th line: head -7 input.txt | tail -1 When it comes to performance, there is not much difference for smaller sizes, but it will be outperformed by the tail | head (from above) when the files become huge. The top-voted sed 'NUMq;d' is interesting to know, but I would argue that it will be understood by fewer people out of the box than the head/tail solution and it is also slower than tail/head. In my tests, both tails/heads versions outperformed sed 'NUMq;d' consistently. That is in line with the other benchmarks that were posted. It is hard to find a case where tails/heads was really bad. It is also not surprising, as these are operations that you would expect to be heavily optimized in a modern Unix system. To get an idea about the performance differences, these are the number that I get for a huge file (9.3G): tail -n+N | head -1: 3.7 sec head -N | tail -1: 4.6 sec sed Nq;d: 18.8 sec Results may differ, but the performance head | tail and tail | head is, in general, comparable for smaller inputs, and sed is always slower by a significant factor (around 5x or so). To reproduce my benchmark, you can try the following, but be warned that it will create a 9.3G file in the current working directory: #!/bin/bash readonly file=tmp-input.txt readonly size=1000000000 readonly pos=500000000 readonly retries=3 seq 1 $size > $file echo "*** head -N | tail -1 ***" for i in $(seq 1 $retries) ; do time head "-$pos" $file | tail -1 done echo "-------------------------" echo echo "*** tail -n+N | head -1 ***" echo seq 1 $size > $file ls -alhg $file for i in $(seq 1 $retries) ; do time tail -n+$pos $file | head -1 done echo "-------------------------" echo echo "*** sed Nq;d ***" echo seq 1 $size > $file ls -alhg $file for i in $(seq 1 $retries) ; do time sed $pos'q;d' $file done /bin/rm $file Here is the output of a run on my machine (ThinkPad X1 Carbon with an SSD and 16G of memory). I assume in the final run everything will come from the cache, not from disk: *** head -N | tail -1 *** 500000000 real 0m9,800s user 0m7,328s sys 0m4,081s 500000000 real 0m4,231s user 0m5,415s sys 0m2,789s 500000000 real 0m4,636s user 0m5,935s sys 0m2,684s ------------------------- *** tail -n+N | head -1 *** -rw-r--r-- 1 phil 9,3G Jan 19 19:49 tmp-input.txt 500000000 real 0m6,452s user 0m3,367s sys 0m1,498s 500000000 real 0m3,890s user 0m2,921s sys 0m0,952s 500000000 real 0m3,763s user 0m3,004s sys 0m0,760s ------------------------- *** sed Nq;d *** -rw-r--r-- 1 phil 9,3G Jan 19 19:50 tmp-input.txt 500000000 real 0m23,675s user 0m21,557s sys 0m1,523s 500000000 real 0m20,328s user 0m18,971s sys 0m1,308s 500000000 real 0m19,835s user 0m18,830s sys 0m1,004s Share Improve this answer Follow edited Jan 20 '18 at 2:19 answered Jul 31 '17 at 13:10 Philipp ClaßenPhilipp Claßen 31k1717 gold badges121121 silver badges187187 bronze badges 2 Is performance different between head | tail vs tail | head? Or does it depend on which line is being printed (beginning of file vs end of file)? – wisbucky Jan 18 '18 at 20:59 1 @wisbucky I have no hard figures, but one disadvantage of first using tail followed by a "head -1" is that you need to know the total length in advance. If you do not know it, you would have to count it first, which will be a loss performance-wise. Another disadvantage is that it is less intuitive to use. For instance, if you have the number 1 to 10 and you want to get the 3rd line, you would have to use "tail -8 | head -1". That is more error prone than "head -3 | tail -1". – Philipp Claßen Jan 18 '18 at 21:11 sorry, I should have included an example to be clear. head -5 | tail -1 vs tail -n+5 | head -1. Actually, I found another answer that did a test comparison and found tail | head to be faster. stackoverflow.com/a/48189289 – wisbucky Jan 19 '18 at 0:28 1 @wisbucky Thank you for mentioning it! I did some tests and have to agree that it was always slightly faster, independent of the position of the line from what I saw. Given that, I changed my answer and also included the benchmark in case someone wants to reproduce it. – Philipp Claßen Jan 19 '18 at 19:15 add a comment  |  27 Wow, all the possibilities! Try this: sed -n "${lineNum}p" $file or one of these depending upon your version of Awk: awk -vlineNum=$lineNum 'NR == lineNum {print $0}' $file awk -v lineNum=4 '{if (NR == lineNum) {print $0}}' $file awk '{if (NR == lineNum) {print $0}}' lineNum=$lineNum $file (You may have to try the nawk or gawk command). Is there a tool that only does the print that particular line? Not one of the standard tools. However, sed is probably the closest and simplest to use. Share Improve this answer Follow edited Aug 8 '13 at 14:06 Kevin 46.8k1212 gold badges9090 silver badges126126 bronze badges answered May 17 '11 at 12:52 David W.David W. 97.6k3434 gold badges203203 silver badges315315 bronze badges add a comment  |  26 # print line number 52 sed '52!d' file Useful one-line scripts for sed Share Improve this answer Follow edited Jan 10 '14 at 15:27 answered Oct 18 '12 at 5:54 Steven PennySteven Penny 78.3k4747 gold badges302302 silver badges345345 bronze badges add a comment  |  21 This question being tagged Bash, here's the Bash (≥4) way of doing: use mapfile with the -s (skip) and -n (count) option. If you need to get the 42nd line of a file file: mapfile -s 41 -n 1 ary < file At this point, you'll have an array ary the fields of which containing the lines of file (including the trailing newline), where we have skipped the first 41 lines (-s 41), and stopped after reading one line (-n 1). So that's really the 42nd line. To print it out: printf '%s' "${ary[0]}" If you need a range of lines, say the range 42–666 (inclusive), and say you don't want to do the math yourself, and print them on stdout: mapfile -s $((42-1)) -n $((666-42+1)) ary < file printf '%s' "${ary[@]}" If you need to process these lines too, it's not really convenient to store the trailing newline. In this case use the -t option (trim): mapfile -t -s $((42-1)) -n $((666-42+1)) ary < file # do stuff printf '%s\n' "${ary[@]}" You can have a function do that for you: print_file_range() { # $1-$2 is the range of file $3 to be printed to stdout local ary mapfile -s $(($1-1)) -n $(($2-$1+1)) ary < "$3" printf '%s' "${ary[@]}" } No external commands, only Bash builtins! Share Improve this answer Follow edited May 17 '14 at 20:35 answered May 17 '14 at 13:45 gniourf_gniourfgniourf_gniourf 37.7k88 gold badges7878 silver badges9191 bronze badges add a comment  |  11 You may also used sed print and quit: sed -n '10{p;q;}' file # print line 10 Share Improve this answer Follow edited Oct 23 '15 at 22:57 Steven Penny 78.3k4747 gold badges302302 silver badges345345 bronze badges answered May 17 '11 at 11:49 berndbernd 11122 bronze badges 6 The -n option disables the default action to print every line, as surely you would have found out by a quick glance at the man page. – tripleee Aug 8 '13 at 14:03 In GNU sed all the sed answers are about the same speed. Therefore (for GNU sed) this is the best sed answer, since it would save time for large files and small nth line values. – agc Apr 2 '17 at 18:08 add a comment  |  9 You can also use Perl for this: perl -wnl -e '$.== NUM && print && exit;' some.file Share Improve this answer Follow edited Oct 21 '14 at 22:07 Toby 22511 silver badge88 bronze badges answered May 16 '11 at 19:43 Timofey StolbovTimofey Stolbov 3,98433 gold badges3535 silver badges4343 bronze badges While testing on a file with 6,000,000 lines, and retrieving arbitrary line #2,000,000, this command was almost instantaneous and much faster than the sed answers. – NoCake Sep 14 '20 at 18:51 add a comment  |  7 As a followup to CaffeineConnoisseur's very helpful benchmarking answer... I was curious as to how fast the 'mapfile' method was compared to others (as that wasn't tested), so I tried a quick-and-dirty speed comparison myself as I do have bash 4 handy. Threw in a test of the "tail | head" method (rather than head | tail) mentioned in one of the comments on the top answer while I was at it, as folks are singing its praises. I don't have anything nearly the size of the testfile used; the best I could find on short notice was a 14M pedigree file (long lines that are whitespace-separated, just under 12000 lines). Short version: mapfile appears faster than the cut method, but slower than everything else, so I'd call it a dud. tail | head, OTOH, looks like it could be the fastest, although with a file this size the difference is not all that substantial compared to sed. $ time head -11000 [filename] | tail -1 [output redacted] real 0m0.117s $ time cut -f11000 -d$'\n' [filename] [output redacted] real 0m1.081s $ time awk 'NR == 11000 {print; exit}' [filename] [output redacted] real 0m0.058s $ time perl -wnl -e '$.== 11000 && print && exit;' [filename] [output redacted] real 0m0.085s $ time sed "11000q;d" [filename] [output redacted] real 0m0.031s $ time (mapfile -s 11000 -n 1 ary < [filename]; echo ${ary[0]}) [output redacted] real 0m0.309s $ time tail -n+11000 [filename] | head -n1 [output redacted] real 0m0.028s Hope this helps! Share Improve this answer Follow edited Jan 10 '18 at 14:16 answered Jan 10 '18 at 14:11 Jo Valentine-CooperJo Valentine-Cooper 11111 silver badge44 bronze badges add a comment  |  6 The fastest solution for big files is always tail|head, provided that the two distances: from the start of the file to the starting line. Lets call it S the distance from the last line to the end of the file. Be it E are known. Then, we could use this: mycount="$E"; (( E > S )) && mycount="+$S" howmany="$(( endline - startline + 1 ))" tail -n "$mycount"| head -n "$howmany" howmany is just the count of lines required. Some more detail in https://unix.stackexchange.com/a/216614/79743 Share Improve this answer Follow edited Apr 13 '17 at 12:36 Community♦ 111 silver badge answered Jul 17 '15 at 5:34 user2350426user2350426 1 Please clarify the units of S and E, (i.e. bytes, chars, or lines). – agc Apr 2 '17 at 18:16 add a comment  |  6 All the above answers directly answer the question. But here's a less direct solution but a potentially more important idea, to provoke thought. Since line lengths are arbitrary, all the bytes of the file before the nth line need to be read. If you have a huge file or need to repeat this task many times, and this process is time-consuming, then you should seriously think about whether you should be storing your data in a different way in the first place. The real solution is to have an index, e.g. at the start of the file, indicating the positions where the lines begin. You could use a database format, or just add a table at the start of the file. Alternatively create a separate index file to accompany your large text file. e.g. you might create a list of character positions for newlines: awk 'BEGIN{c=0;print(c)}{c+=length()+1;print(c+1)}' file.txt > file.idx then read with tail, which actually seeks directly to the appropriate point in the file! e.g. to get line 1000: tail -c +$(awk 'NR=1000' file.idx) file.txt | head -1 This may not work with 2-byte / multibyte characters, since awk is "character-aware" but tail is not. I haven't tested this against a large file. Also see this answer. Alternatively - split your file into smaller files! Share Improve this answer Follow answered Oct 12 '17 at 10:44 Sanjay ManoharSanjay Manohar 6,56511 gold badge3131 silver badges5454 bronze badges add a comment  |  4 Using what others mentioned, I wanted this to be a quick & dandy function in my bash shell. Create a file: ~/.functions Add to it the contents: getline() { line=$1 sed $line'q;d' $2 } Then add this to your ~/.bash_profile: source ~/.functions Now when you open a new bash window, you can just call the function as so: getline 441 myfile.txt Share Improve this answer Follow answered Jan 17 '18 at 14:19 Mark Shust at M.academyMark Shust at M.academy 5,37333 gold badges2626 silver badges4444 bronze badges There is no need to assign $1 to another variable before using it, and you are clobbering any other global line. In Bash, use local for function variables; but here, as stated already, probably just do sed "$1d;q" "$2". (Notice also the quoting of "$2".) – tripleee Oct 19 '20 at 11:22 Correct, but it could be helpful to have self-documented code. – Mark Shust at M.academy Oct 20 '20 at 13:10 add a comment  |  3 If you got multiple lines by delimited by \n (normally new line). You can use 'cut' as well: echo "$data" | cut -f2 -d$'\n' You will get the 2nd line from the file. -f3 gives you the 3rd line. Share Improve this answer Follow answered Jan 7 '16 at 16:27 danger89danger89 2,1511616 silver badges2424 bronze badges 1 Can be also used to display multiple lines: cat FILE | cut -f2,5 -d$'\n' will display lines 2 and 5 of the FILE. (But it will not preserve the order.) – Andriy Makukha Jan 25 '18 at 11:16 add a comment  |  3 Lots of good answers already. I personally go with awk. For convenience, if you use bash, just add the below to your ~/.bash_profile. And, the next time you log in (or if you source your .bash_profile after this update), you will have a new nifty "nth" function available to pipe your files through. Execute this or put it in your ~/.bash_profile (if using bash) and reopen bash (or execute source ~/.bach_profile) # print just the nth piped in line nth () { awk -vlnum=${1} 'NR==lnum {print; exit}'; } Then, to use it, simply pipe through it. E.g.,: $ yes line | cat -n | nth 5 5 line Share Improve this answer Follow edited Oct 20 '20 at 13:45 tripleee 133k2323 gold badges195195 silver badges257257 bronze badges answered Nov 17 '17 at 15:42 JJCJJC 7,02555 gold badges3939 silver badges4949 bronze badges add a comment  |  1 To print nth line using sed with a variable as line number: a=4 sed -e $a'q:d' file Here the '-e' flag is for adding script to command to be executed. Share Improve this answer Follow edited Apr 7 '15 at 18:59 answered Mar 13 '15 at 6:40 aliasavaliasav 2,53522 gold badges2121 silver badges3030 bronze badges 2 The colon is a syntax error, and should be a semicolon. – tripleee Feb 16 '16 at 4:26 add a comment  |  1 After taking a look at the top answer and the benchmark, I've implemented a tiny helper function: function nth { if (( ${#} < 1 || ${#} > 2 )); then echo -e "usage: $0 \e[4mline\e[0m [\e[4mfile\e[0m]" return 1 fi if (( ${#} > 1 )); then sed "$1q;d" $2 else sed "$1q;d" fi } Basically you can use it in two fashions: nth 42 myfile.txt do_stuff | nth 42 Share Improve this answer Follow answered May 12 '20 at 8:45 Ulysse BNUlysse BN 5,88022 gold badges3030 silver badges5757 bronze badges add a comment  |  0 I've put some of the above answers into a short bash script that you can put into a file called get.sh and link to /usr/local/bin/get (or whatever other name you prefer). #!/bin/bash if [ "${1}" == "" ]; then echo "error: blank line number"; exit 1 fi re='^[0-9]+$' if ! [[ $1 =~ $re ]] ; then echo "error: line number arg not a number"; exit 1 fi if [ "${2}" == "" ]; then echo "error: blank file name"; exit 1 fi sed "${1}q;d" $2; exit 0 Ensure it's executable with $ chmod +x get Link it to make it available on the PATH with $ ln -s get.sh /usr/local/bin/get Enjoy responsibly! P Share Improve this answer Follow answered Jan 28 '19 at 15:22 polarisepolarise 1,94511 gold badge1717 silver badges2424 bronze badges add a comment  |  Highly active question. Earn 10 reputation in order to answer this question. The reputation requirement helps protect this question from spam and non-answer activity. Not the answer you're looking for? Browse other questions tagged bash shell unix awk sed or ask your own question. The Overflow Blog A deeper dive into our May 2019 security incident Podcast 307: Owning the code, from integration to delivery Featured on Meta Opt-in alpha test for a new Stacks editor 2020: a year in moderation Linked 16 How do I read the Nth line of a file and print it to a new file? 1 Get a line from a file and append it to another file 0 How to save a specified line of a file to a variable 1 How to get the 'variable' line from file? 0 SSH to server read from file -1 How to Save 'specific' line from terminal output to file? 0 How to get the nth line from a file and save the result to a variable -1 How to print the second line of a 3 line pattern in a text file using unix tool? 0 Crontab edit cron tasks within terminal 0 How to extract the third largest file size in Unix See more linked questions Related 5246 How can I get the source directory of a Bash script from within the script itself? 2078 How do I parse command line arguments in Bash? 2399 How can I check if a program exists from a Bash script? 3431 How do I tell if a regular file does not exist in Bash? 1612 How to redirect and append both stdout and stderr to a file with Bash? 1531 Looping through the content of a file in Bash 556 How to split a large text file into smaller files with equal number of lines? 1053 How to reload .bash_profile from the command line? 1912 How to delete from a text file, all lines that contain a specific string? Hot Network Questions How can I convert a JPEG image to a RAW image with a Linux command? Did Barry Goldwater claim peanut butter is good shaving cream? Coughing fits in cats How much is 1 mμg? I am confused about modes? Can someone explain it in these terms Finding a proper adverb to end a sentence meaning unnecessary but not otherwise a problem Biking Target Speeds Order of operations and rounding for microcontrollers Can the Wish spell change my spell list? Why are they so good at magic? Inserting © (copyright symbol) using Microsoft Word Can someone tell me the purpose of this multi-tool? How are probabilities written in German? Expectations from a violin teacher towards an adult learner Implement a zipwith function How can I motivate the teaching assistants to grade more strictly? What is an effective way to evaluate and assess employees on a non-management career track? I need drivers for Linux install, on my old laptop, Because my laptop is old, will there be any problem if I install Linux? Are all atoms spherically symmetric? If so, why are atoms with half-filled/filled sub-shells often quoted as 'especially' spherically symmetric? Put Option shown as negative value What happen if a Twilight Domain Cleric that shared his darkvision becomes unconscious or dies? Why do some people argue that contingency fees increase lawsuits? Why is KID considered more sound than Pirc? How did US Supreme court Justice John Roberts "let it be known" that he did not wish to preside over the 2nd presidential impeachment? more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. lang-sh Stack Overflow works best with JavaScript enabled
Join Stack Overflow to learn, share knowledge, and build your career. Sign up with email Sign up Sign up with Google Sign up with GitHub Sign up with Facebook Teams Q&A for Work Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information. Learn more Bash, how to hash value of a string? Ask Question Asked 5 years, 11 months ago Active 5 years, 11 months ago Viewed 10k times 9 1 I want to simply convert a string of any length to an integer value. Each string will map to a unique or even non-unique integer. Is there any existing opensource command that does this? Bonus points if it is unique, such as computing the lexicographical order via a bash command. string bash Share Improve this question Follow edited Mar 4 '15 at 0:30 Zombies asked Mar 4 '15 at 0:14 ZombiesZombies 22.5k3737 gold badges131131 silver badges213213 bronze badges 1 Must be an integer, or any unique hash will suffice? – bishop Mar 4 '15 at 0:18 Integer only (not the datatype, it can be a "long" or "big" int). So just to be clear, it can be a string that has any length of digits in it. EG: [0-9]+ – Zombies Mar 4 '15 at 0:22 2 I am interested to see how you give bonus points… – kojiro Mar 4 '15 at 4:41 I would mark it as accepted over the other answers. But since rici's answer has multiple ways of doing this, each with highlighted differences, I marked his as correct. – Zombies Mar 30 '15 at 14:07 add a comment  |  2 Answers 2 Active Oldest Votes 14 You need to be careful about using hash functions from common programming languages. It has been common to introduce randomized seeds into hash functions, so that hash values are only unique for a single program execution. This avoids a denial-of-service attack noted in oCert advisory 2011-3. (As that advisory notes, the problem was described in 2003 in a paper presented to Usenix.) For example, the Python hash function has been randomized by default since v3.3: $ python3 -c 'from sys import argv;print(hash(argv[1]))' abc -2595772619214671013 $ python3 -c 'from sys import argv;print(hash(argv[1]))' abc -6001956461950650533 $ python3 -c 'from sys import argv;print(hash(argv[1]))' abc -7414807274805087300 $ python3 -c 'from sys import argv;print(hash(argv[1]))' abc -327608370992723225 # Python2 generates consistent hash values $ python -c 'from sys import argv;print(hash(argv[1]))' abc 1453079729188098211 $ python -c 'from sys import argv;print(hash(argv[1]))' abc 1453079729188098211 $ python -c 'from sys import argv;print(hash(argv[1]))' abc 1453079729188098211 You can control hash randomization in Python by setting the PYTHONHASHSEED environment variable. Or you can use a standardized cryptographic hash like SHA-1. The commonly-available sha1sum utility outputs its result in hexadecimal, but you can convert that to decimal with bash (truncated to 64 bits): $ echo $((0x$(sha1sum <<<"string to hash")0)) -7037254581539467098 or in its full 160-bit glory, using bc (which requires hex to be written in upper-case): $ bc <<<ibase=16\;$(sha1sum <<<"string to hash"|tr a-z A-Z)0 861191872165666513280590001082621748432296579238 If you only need the hash value modulo some power of 16, you can use the first few bytes of the SHA-1 sum. (You could use any selection of bytes -- they're all equally well distributed -- but the first few are easier to extract): $ echo $((0x$(sha1sum <<<"string to hash"|cut -c1-2))) 150 Note: As @gniourf_gniourf points out in a comment, the above doesn't really compute the SHA-1 checksum of the given string because the bash here-string syntax (<<<word) appends a newline to word. Since the checksum of the string with a newline appended is just as good a hash as the checksum of the string itself, there is no problem as long as you always use the same mechanism to produce the hash. Share Improve this answer Follow edited May 23 '17 at 12:01 Community♦ 111 silver badge answered Mar 4 '15 at 4:26 ricirici 195k2323 gold badges189189 silver badges274274 bronze badges 1 A sidenote: here string adds a trailing newline. So this doesn't compute the sha1sum of the string, but of the string plus a trailing newline. It doesn't matter here, but if you really want the sha1sum: sha1sum < <(printf '%s' "string to hash"). – gniourf_gniourf Mar 4 '15 at 7:03 Things of beauty! That bc one is my favorite. – bishop Mar 4 '15 at 15:26 add a comment  |  12 You could use the sum or cksum command (the latter being preferred) to generate a base-10 integer: $ cksum <<< 'hello world' | cut -f 1 -d ' ' 3733384285 $ cksum <<< 'goodbye world' | cut -f 1 -d ' ' 2600070097 If you're interested in the math behind these simple hashes, check out the source implementations: cksum calculates the AUTODIN II polynomial used by Ethernet sum calculates either the 16-bit CRC or the POSIX 1003.2 CRC, depending upon the -r and -s command-line arguments. Share Improve this answer Follow edited Mar 4 '15 at 15:25 answered Mar 4 '15 at 0:32 bishopbishop 31.5k88 gold badges8888 silver badges120120 bronze badges 1 Exactly what I was looking for. – Shoan Oct 14 '16 at 5:19 add a comment  |  Not the answer you're looking for? Browse other questions tagged string bash or ask your own question. The Overflow Blog A deeper dive into our May 2019 security incident Podcast 307: Owning the code, from integration to delivery Featured on Meta Opt-in alpha test for a new Stacks editor 2020: a year in moderation Related 6802 What is the difference between String and string in C#? 3883 How can I check if a directory exists in a Bash shell script? 5246 How can I get the source directory of a Bash script from within the script itself? 2797 How to check if a string contains a substring in Bash 4245 How do I read / convert an InputStream into a String in Java? 3431 How do I tell if a regular file does not exist in Bash? 2260 Extract filename and extension in Bash 4682 How to replace all occurrences of a string in Javascript? 7421 How to check whether a string contains a substring in JavaScript? 2971 How to concatenate string variables in Bash Hot Network Questions OpenStreetMap baselayer of just one country Why don't flights fly towards their landing approach path sooner? Why don't video conferencing web applications ask permission for screen sharing? Can someone tell me the purpose of this multi-tool? How can I convert a JPEG image to a RAW image with a Linux command? How can I motivate the teaching assistants to grade more strictly? What happened to the Cross Particles stolen by Mitch Carson? How are probabilities written in German? Put Option shown as negative value Is it a lobster number? Hardness of a problem which is the sum of two NP-Hard problems Why people choose 0.2 as the value of linking length in the friends-of-friends algorithm? Proof that a Cartesian category is monoidal How to reply to students' emails that show anger about their mark? Why isn't SpaceX's Starship trial and error great and unique development strategy an opensource project? What is a good noun to refer to somebody who is unhappy IP resolves to strange domain Why does find not find my directory neither with -name nor with -regex How likely it is that a nobleman of the eighteenth century would give written instructions to his maids? Can I be a good scientist if I only work in working hours? Where were mathematical/science works posted before the arxiv website? Skyscrapers 6x6 - Stuck at the end Is mirror test a good way to explore alien inhabited world safely? How does rubbing soap on wet skin produce foam, and does it really enhance cleaning? more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. lang-sh Stack Overflow works best with JavaScript enabled
Join Stack Overflow to learn, share knowledge, and build your career. Sign up with email Sign up Sign up with Google Sign up with GitHub Sign up with Facebook Teams Q&A for Work Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information. Learn more Command to get time in milliseconds Ask Question Asked 7 years, 8 months ago Active 5 months ago Viewed 393k times 325 78 Is there a shell command in Linux to get the time in milliseconds? linux bash shell time Share Improve this question Follow edited Mar 28 '18 at 23:11 codeforester 27.3k99 gold badges7373 silver badges103103 bronze badges asked May 14 '13 at 16:42 MOHAMEDMOHAMED 34.8k4848 gold badges139139 silver badges235235 bronze badges 15 date +%s.%N would give you nanoseconds, can you work with that? – Wrikken May 14 '13 at 17:01 1 @Wrikken This is not quite portable though. – Camilo Martin Jun 23 '14 at 12:43 17 date +"%Y%m%d.%H%M%S%3N" - Generally i use this command to get the fulldate and unique time till milli seconds to create the temporary file names in unix bash script. If your system is not able to cope up milli seconds you can go to micro or nano seconds using 3 to 6 and 9 respectively. – Nitin Mahesh Jan 14 '16 at 3:51 Try date -Ins – SO Stinks Feb 5 '20 at 10:30 add a comment  |  12 Answers 12 Active Oldest Votes 375 date +%s%N returns the number of seconds + current nanoseconds. Therefore, echo $(($(date +%s%N)/1000000)) is what you need. Example: $ echo $(($(date +%s%N)/1000000)) 1535546718115 date +%s returns the number of seconds since the epoch, if that's useful. Share Improve this answer Follow edited Nov 18 '19 at 13:09 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered May 14 '13 at 17:00 AlperAlper 11.1k22 gold badges2828 silver badges3636 bronze badges 78 doesn't work on Mac OS, since %N is not supported by date – yegor256 Aug 27 '13 at 18:36 35 Question is asking for Linux command. @alper's answer works fine for date command with GNU coreutils. GNUtize your OSX: Install and Use GNU Command Line Tools on Mac OS X – caligari Feb 21 '14 at 7:58 83 date +%s%3N is faster (based in @michael-defort's answer) – caligari Feb 21 '14 at 8:08 12 On OSX you need to install the GNU version of date as part of coreutils using MacPorts or Homebrew - then use the gdate command. See this question: stackoverflow.com/questions/9804966/… – Pierz Jun 23 '15 at 22:10 1 Although date +%s%3N seems to be easier or better, but using it in some other offset calculation caused the timestamp to be reduced by 1 millisecond! But this solution worked perfect with offset calculation – Arsinux Apr 20 '18 at 12:59  |  show 3 more comments 389 date +"%T.%N" returns the current time with nanoseconds. 06:46:41.431857000 date +"%T.%6N" returns the current time with nanoseconds rounded to the first 6 digits, which is microseconds. 06:47:07.183172 date +"%T.%3N" returns the current time with nanoseconds rounded to the first 3 digits, which is milliseconds. 06:47:42.773 In general, every field of the date command's format can be given an optional field width. Share Improve this answer Follow edited Aug 29 '18 at 12:49 rogerdpack 48.5k3131 gold badges205205 silver badges325325 bronze badges answered Sep 9 '13 at 17:55 Michael DefortMichael Defort 3,89911 gold badge99 silver badges22 bronze badges 37 %xN: nice one for the field width! – fduff Jan 22 '14 at 12:47 1 date +"%Y%m%d.%H%M%S%3N" for milli seconds. – Nitin Mahesh Jan 14 '16 at 3:48 5 I guess this is a better answer that the one that is marked as the right one. – kcondezo Apr 19 '18 at 14:26 add a comment  |  83 Nano is 10−9 and milli 10−3. Hence, we can use the three first characters of nanoseconds to get the milliseconds: date +%s%3N From man date: %N nanoseconds (000000000..999999999) %s seconds since 1970-01-01 00:00:00 UTC Source: Server Fault's How do I get the current Unix time in milliseconds in Bash?. Share Improve this answer Follow edited Nov 18 '19 at 13:18 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Jun 24 '15 at 7:49 fedorqui 'SO stop harming'fedorqui 'SO stop harming' 221k7676 gold badges450450 silver badges505505 bronze badges I feel so proud that @Peter Mortensen did one of his superb active readings on one of my posts :) thanks!! – fedorqui 'SO stop harming' Nov 18 '19 at 13:44 add a comment  |  60 On OS X, where date does not support the %N flag, I recommend installing coreutils using Homebrew. This will give you access to a command called gdate that will behave as date does on Linux systems. brew install coreutils For a more "native" experience, you can always add this to your .bash_aliases: alias date='gdate' Then execute $ date +%s%N Share Improve this answer Follow edited Nov 18 '19 at 13:20 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Nov 10 '15 at 23:21 Joshua CookJoshua Cook 7,53022 gold badges3030 silver badges2828 bronze badges add a comment  |  14 Here is a somehow portable hack for Linux for getting time in milliseconds: #!/bin/sh read up rest </proc/uptime; t1="${up%.*}${up#*.}" sleep 3 # your command read up rest </proc/uptime; t2="${up%.*}${up#*.}" millisec=$(( 10*(t2-t1) )) echo $millisec The output is: 3010 This is a very cheap operation, which works with shell internals and procfs. Share Improve this answer Follow edited Nov 18 '19 at 13:48 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Feb 13 '18 at 10:53 Bastian BittorfBastian Bittorf 31722 silver badges66 bronze badges 2 Only one so far which worked on the Xeon Phi BusyBox v1.27.0 (2017-09-27 13:20:28 EDT) MicroOS - would gladly upvote three times! – Cadoiz May 7 '19 at 3:00 I'm using BusyBox 1.27.1. Also the only solution to work. – Jim Fell Nov 9 '20 at 16:49 add a comment  |  9 date command didnt provide milli seconds on OS X, so used an alias from python millis(){ python -c "import time; print(int(time.time()*1000))"; } OR alias millis='python -c "import time; print(int(time.time()*1000))"' EDIT: following the comment from @CharlesDuffy. Forking any child process takes extra time. $ time date +%s%N 1597103627N date +%s%N 0.00s user 0.00s system 63% cpu 0.006 total Python is still improving it's VM start time, and it is not as fast as ahead-of-time compiled code (such as date). On my machine, it took about 30ms - 60ms (that is 5x-10x of 6ms taken by date) $ time python -c "import time; print(int(time.time()*1000))" 1597103899460 python -c "import time; print(int(time.time()*1000))" 0.03s user 0.01s system 83% cpu 0.053 total I figured awk is lightweight than python, so awk takes in the range of 6ms to 12ms (i.e. 1x to 2x of date): $ time awk '@load "time"; BEGIN{print int(1000 * gettimeofday())}' 1597103729525 awk '@load "time"; BEGIN{print int(1000 * gettimeofday())}' 0.00s user 0.00s system 74% cpu 0.010 total Share Improve this answer Follow edited Aug 11 '20 at 0:00 answered Oct 16 '16 at 7:29 Thamme GowdaThamme Gowda 9,15111 gold badge4040 silver badges4949 bronze badges 4 ...however, once you've fork()ed off a separate process, execed your Python interpreter, let it load its libraries / otherwise initialize, write its result, and exit, that result will no longer be accurate. – Charles Duffy Apr 30 '19 at 0:46 add a comment  |  6 The other answers are probably sufficient in most cases but I thought I'd add my two cents as I ran into a problem on a BusyBox system. The system in question did not support the %N format option and doesn't have no Python or Perl interpreter. After much head scratching, we (thanks Dave!) came up with this: adjtimex | awk '/(time.tv_sec|time.tv_usec):/ { printf("%06d", $2) }' It extracts the seconds and microseconds from the output of adjtimex (normally used to set options for the system clock) and prints them without new lines (so they get glued together). Note that the microseconds field has to be pre-padded with zeros, but this doesn't affect the seconds field which is longer than six digits anyway. From this it should be trivial to convert microseconds to milliseconds. If you need a trailing new line (maybe because it looks better) then try adjtimex | awk '/(time.tv_sec|time.tv_usec):/ { printf("%06d", $2) }' && printf "\n" Also note that this requires adjtimex and awk to be available. If not then with BusyBox you can point to them locally with: ln -s /bin/busybox ./adjtimex ln -s /bin/busybox ./awk And then call the above as ./adjtimex | ./awk '/(time.tv_sec|time.tv_usec):/ { printf("%06d", $2) }' Or of course you could put them in your PATH EDIT: The above worked on my BusyBox device. On Ubuntu I tried the same thing and realised that adjtimex has different versions. On Ubuntu this worked to output the time in seconds with decimal places to microseconds (including a trailing new line) sudo apt-get install adjtimex adjtimex -p | awk '/raw time:/ { print $6 }' I wouldn't do this on Ubuntu though. I would use date +%s%N Share Improve this answer Follow edited Nov 18 '19 at 13:30 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Aug 10 '16 at 11:32 pghallidaypghalliday 13111 silver badge66 bronze badges Wow, great alternative! Indeed your command works, but I don't have a clue why. Where do I find a documentation for the awk command?! How in hell did you find out how to build the string to extract the desired information out of the adjtimex output? – Satria Oct 26 '18 at 23:28 Awesome busybox solution – Codebling Jan 25 '19 at 22:58 If there is a possiblity to compile Busybox yourself, enabling "CONFIG_FEATURE_DATE_NANO" would be another approach. – Bl00dh0und Aug 10 '20 at 11:21 add a comment  |  2 I just wanted to add to Alper's answer what I had to do to get this stuff working: On Mac, you'll need brew install coreutils, so we can use gdate. Otherwise on Linux, it's just date. And this function will help you time commands without having to create temporary files or anything: function timeit() { start=`gdate +%s%N` bash -c $1 end=`gdate +%s%N` runtime=$(((end-start)/1000000000.0)) echo " seconds" } And you can use it with a string: timeit 'tsc --noEmit' Share Improve this answer Follow edited Nov 18 '19 at 13:49 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered May 15 '17 at 18:47 ChetChet 14k1313 gold badges5656 silver badges9999 bronze badges I like this solution but I'm more interested in millis. I ported this function to my .bashrc in ubuntu: function timeit () { start=$(date +%s%N); $*; end=$(date +%s%N); runtime=$(((end-start)/1000000)); echo "$runtime ms" } – Uluaiv Jan 30 '20 at 12:45 add a comment  |  1 Perl can be used for this, even on exotic platforms like AIX. Example: #!/usr/bin/perl -w use strict; use Time::HiRes qw(gettimeofday); my ($t_sec, $usec) = gettimeofday (); my $msec= int ($usec/1000); my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime ($t_sec); printf "%04d-%02d-%02d %02d:%02d:%02d %03d\n", 1900+$year, 1+$mon, $mday, $hour, $min, $sec, $msec; Share Improve this answer Follow answered Nov 28 '18 at 11:20 Lorinczy ZsigmondLorinczy Zsigmond 1,33911 gold badge1010 silver badges1919 bronze badges add a comment  |  1 A Python script like this: import time cur_time = int(time.time()*1000) Share Improve this answer Follow edited Nov 18 '19 at 13:14 Peter Mortensen 27.5k2121 gold badges9393 silver badges123123 bronze badges answered Jun 23 '15 at 22:09 maoyangmaoyang 90111 gold badge1010 silver badges1010 bronze badges 3 this will not return the number of milliseconds, this will return the number of seconds expressed in milliseconds. Everything will be $SECONDS000 – kilianc Nov 20 '15 at 19:08 1 @kilianc maoyang's Python code is providing milliseconds. – jlliagre Sep 28 '16 at 12:53 @jlliagre: But is the actual time resolution better than 16-17 ms (1/60 second) or not? – Peter Mortensen Nov 18 '19 at 13:15 Yes, Python's time.time() returns a float to 6 decimal places, at least on macOS. – porglezomp Dec 18 '19 at 3:49 add a comment  |  1 When you use GNU AWK since version 4.1, you can load the time library and do: $ awk '@load "time"; BEGIN{printf "%.6f", gettimeofday()}' This will print the current time in seconds since 1970-01-01T00:00:00 in sub second accuracy. the_time = gettimeofday() Return the time in seconds that has elapsed since 1970-01-01 UTC as a floating-point value. If the time is unavailable on this platform, return -1 and set ERRNO. The returned time should have sub-second precision, but the actual precision may vary based on the platform. If the standard C gettimeofday() system call is available on this platform, then it simply returns the value. Otherwise, if on MS-Windows, it tries to use GetSystemTimeAsFileTime(). source: GNU awk manual On Linux systems, the standard C function getimeofday() returns the time in microsecond accuracy. Share Improve this answer Follow edited Nov 18 '19 at 13:56 answered Nov 18 '19 at 13:24 kvantourkvantour 19.9k44 gold badges3636 silver badges4747 bronze badges specifically GAWK, right? I tried it on regular AWK, and it failed. – Sapphire_Brick Nov 25 '20 at 2:27 add a comment  |  0 To show date with time and time-zone date +"%d-%m-%Y %T.%N %Z" Output : 22-04-2020 18:01:35.970289239 IST Share Improve this answer Follow answered Apr 22 '20 at 12:33 Anil AgrawalAnil Agrawal 1,6741919 silver badges2525 bronze badges add a comment  |  Highly active question. Earn 10 reputation in order to answer this question. The reputation requirement helps protect this question from spam and non-answer activity. Not the answer you're looking for? Browse other questions tagged linux bash shell time or ask your own question. The Overflow Blog A deeper dive into our May 2019 security incident Podcast 307: Owning the code, from integration to delivery Featured on Meta Opt-in alpha test for a new Stacks editor 2020: a year in moderation Linked 62 Date command does not follow Linux specifications (Mac OS X Lion) 16 How do I stop set.seed() after a specific line of code? 1 bash - convert date time into compatible format for kibana 2 Time variable address in Linux 0 bash timer in milliseconds 1 Running Linux Shell on AIX 0 convert log4j timestamp to millisecond through unix command Related 3883 How can I check if a directory exists in a Bash shell script? 5246 How can I get the source directory of a Bash script from within the script itself? 5147 How to execute a program or call a system command from Python? 1536 How do I prompt for Yes/No/Cancel input in a Linux shell script? 3133 How to get the current time in Python 2458 In the shell, what does “ 2>&1 ” mean? 1053 How to reload .bash_profile from the command line? 1846 How do I set a variable to the output of a command in Bash? 5708 How do I find all files containing specific text on Linux? Hot Network Questions What type of lightbulb is this? Can I be a good scientist if I only work in working hours? When sharing the Eyes of Night darkvision, does a creature needs to always be 10 feet close to the cleric to be granted the benefits? How Do I Compress Multiple Novels' Worth of Plot, Characters, and Worldbuilding into One? Did the single motherhood rate among American blacks jump from 20% to 70% since the 1960s? How far can a wolf drag a 175 lb human on tundra? Does Hitman 3 contain nudity? Are there any diacritics not on the top or bottom of a letter? How much is 1 mμg? Implement a zipwith function Is it a lobster number? How Can Vampires Obtain Donors? Construct a permutation tree plot Save performance with large update on Index with Included Column Could bug bounty hunting accidentally cause real damage? Why do some people argue that contingency fees increase lawsuits? Why isn't SpaceX's Starship trial and error great and unique development strategy an opensource project? Why people choose 0.2 as the value of linking length in the friends-of-friends algorithm? Why is KID considered more sound than Pirc? Generate random string to match the goal string with minimum iteration and control statements What do "tangential and centripetal acceleration" mean for non-circular motion? Where were mathematical/science works posted before the arxiv website? What is an effective way to evaluate and assess employees on a non-management career track? Frame dropout cracked, what can I do? (Allied Alfa Disc / carbon) more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. lang-sh Stack Overflow works best with JavaScript enabled
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up. Sign up to join this community Anybody can ask a question Anybody can answer The best answers are voted up and rise to the top Display Spinner while waiting for some process to finish Ask Question Asked 5 years, 5 months ago Active 2 months ago Viewed 40k times 18 12 How can I show spinner till command line finish it is job? In other words, If I am running a script and I want to show spinner while this script is running and the spinner disappears when the script finish it is job. Bellow is a common spinner code: i=1 sp="/-\|" echo -n ' ' while true do printf "\b${sp:i++%${#sp}:1}" done How can I link the previous spinner code to a command to let it show spinner while the command is running and the spinner disappears when the command finish it is job? If I include the command inside the loop it will loop with the spinner so what is the solution in this case? bash Share Improve this question Follow asked Aug 24 '15 at 17:03 user88036user88036 add a comment  |  5 Answers 5 Active Oldest Votes 25 Have your while loop watch for your real command to exit. I'll assume a Linux environment that has /proc entries for each PID, but you could slice it other ways: #!/bin/bash # your real command here, instead of sleep sleep 7 & PID=$! i=1 sp="/-\|" echo -n ' ' while [ -d /proc/$PID ] do printf "\b${sp:i++%${#sp}:1}" done Share Improve this answer Follow answered Aug 24 '15 at 17:22 Jeff Schaller♦Jeff Schaller 57.8k1212 gold badges9090 silver badges201201 bronze badges 9 This is a busy loop that will eat up cpu resources. I'd suggest having a delay of some kind in your while loop. – ACase Jul 20 '16 at 14:36 This makes sense and seems to be the simplest method, but it doesn't work for me (on MacOS), so I'm trying to make it work. What does -d /proc/$PID do? – matharden Jul 7 '20 at 11:04 -d /proc/$PID, as part of the test command [ ... ], checks whether /proc/$PID exists as a directory (after expanding the $PID variable, of course). It's a quick way in Linux to see if a process exists with that PID. – Jeff Schaller♦ Jul 7 '20 at 11:49 add a comment  |  19 This shell script should do what you're looking for: #!/usr/bin/env bash show_spinner() { local -r pid="${1}" local -r delay='0.75' local spinstr='\|/-' local temp while ps a | awk '{print $1}' | grep -q "${pid}"; do temp="${spinstr#?}" printf " [%c] " "${spinstr}" spinstr=${temp}${spinstr%"${temp}"} sleep "${delay}" printf "\b\b\b\b\b\b" done printf " \b\b\b\b" } ("$@") & show_spinner "$!" Assuming you store the shell script in a file named spinner, you can invoke it like this to display a spinner while the command sleep 10 is running: $ spinner sleep 10 Share Improve this answer Follow edited Jun 25 '19 at 15:46 answered Apr 14 '16 at 21:03 jsearsjsears 29122 silver badges66 bronze badges 1 See also stackoverflow.com/a/20369590/2908724 – bishop Oct 21 '16 at 17:28 This assumes the shell script is called spinner, so no. I think the example is right, assuming you name the script spinner. – jsears Jun 25 '19 at 15:44 add a comment  |  8 Here's another fancy spinner which you can use like this: spinner ping google.com echo "ping exited with exit code $?" spinner sleep 10 echo "sleep exited with exit code $?" It has 12 themes and picks one randomly. #!/bin/bash # Shows a spinner while another command is running. Randomly picks one of 12 spinner styles. # @args command to run (with any parameters) while showing a spinner. # E.g. ‹spinner sleep 10› function shutdown() { tput cnorm # reset cursor } trap shutdown EXIT function cursorBack() { echo -en "\033[$1D" } function spinner() { # make sure we use non-unicode character type locale # (that way it works for any locale as long as the font supports the characters) local LC_CTYPE=C local pid=$1 # Process Id of the previous running command case $(($RANDOM % 12)) in 0) local spin='⠁⠂⠄⡀⢀⠠⠐⠈' local charwidth=3 ;; 1) local spin='-\|/' local charwidth=1 ;; 2) local spin="▁▂▃▄▅▆▇█▇▆▅▄▃▂▁" local charwidth=3 ;; 3) local spin="▉▊▋▌▍▎▏▎▍▌▋▊▉" local charwidth=3 ;; 4) local spin='←↖↑↗→↘↓↙' local charwidth=3 ;; 5) local spin='▖▘▝▗' local charwidth=3 ;; 6) local spin='┤┘┴└├┌┬┐' local charwidth=3 ;; 7) local spin='◢◣◤◥' local charwidth=3 ;; 8) local spin='◰◳◲◱' local charwidth=3 ;; 9) local spin='◴◷◶◵' local charwidth=3 ;; 10) local spin='◐◓◑◒' local charwidth=3 ;; 11) local spin='⣾⣽⣻⢿⡿⣟⣯⣷' local charwidth=3 ;; esac local i=0 tput civis # cursor invisible while kill -0 $pid 2>/dev/null; do local i=$(((i + $charwidth) % ${#spin})) printf "%s" "${spin:$i:$charwidth}" cursorBack 1 sleep .1 done tput cnorm wait $pid # capture exit code return $? } ("$@") & spinner $! Share Improve this answer Follow edited Feb 20 '20 at 9:15 answered Feb 3 '20 at 14:55 Jonas EberleJonas Eberle 19211 silver badge66 bronze badges Very interesting, but not working at all on macOS zsh. – kitsune Jul 11 '20 at 8:41 it is a bash script. If you chmod +x it and run it via ./script it'll be run with /bin/bash (even if your shell runs zsh). I just tried it (with Linux though) and I don't see why it shouldn't work with MacOS. – Jonas Eberle Jul 12 '20 at 9:59 1 Of course I passed the script under chmod! :D However, the case 1) works fine, but other cases, for example 11) seems that cursorBack() doesn't work, so I get a long line of characters printed in succession. – kitsune Jul 13 '20 at 8:29 Seems not to be a zsh or bash problem I guess. I think your terminal does not know what to do with `echo -en "\033[$1D"`` - maybe we could find a sequence for "cursorBack" which is more portable even for unusual terminal settings? – Jonas Eberle Jul 14 '20 at 11:56 Man.. this is a beauty.. thank you for the inspiration!! – TacB0sS Nov 21 '20 at 0:36 add a comment  |  3 If you want a lowest common denominator spinner that works with /bin/sh and doesn't rely on the extended bash parameter substitution this should work: #!/bin/sh # The command you are waiting on goes between the ( ) here # The example below returns a non zero return code (sleep 20 ; /bin/false) & pid=$! ; i=0 while ps -a | awk '{print $1}' | grep -q "${pid}" do c=`expr ${i} % 4` case ${c} in 0) echo "/\c" ;; 1) echo "-\c" ;; 2) echo "\\ \b\c" ;; 3) echo "|\c" ;; esac i=`expr ${i} + 1` # change the speed of the spinner by altering the 1 below sleep 1 echo "\b\c" done # Collect the return code from the background process wait ${pid} ret=$? # You can report on any errors due to a non zero return code here exit ${ret} Share Improve this answer Follow edited Apr 12 '19 at 20:40 answered Apr 12 '19 at 20:33 Rob BartlettRob Bartlett 3122 bronze badges add a comment  |  0 So while all the above answer work.. I thought to add mine: DEFAULT_SpinnerFrames=("—" "\\" "|" "/") ## @function: spinner(action, label, &spinnerFramesRef[]) ## ## @description: Perform an action asynchronously and display ## spinner till action is completed ## ## @param action: The action the execute ## @param label: The label to display while waiting ## @param spinnerRef: In case you feel like a custom spinner, pass a ref to an array of strings spinner() { local frameRef local action="${1}" local label="${2} " local spinnerRef="${3-DEFAULT_SpinnerFrames}" local spinnerFrames=$(eval "echo \${!${spinnerRef}[@]}") spinnerRun() { while true; do for frame in ${spinnerFrames[@]}; do frameRef="${spinnerRef}[${frame}]" echo "${label}${!frameRef}" tput cuu1 tput el sleep 0.2 done done echo -e "\r" } spinnerRun & local spinnerPid=$! ${action} kill "${spinnerPid}" } github tests Share Improve this answer Follow edited Nov 21 '20 at 19:41 answered Nov 21 '20 at 0:35 TacB0sSTacB0sS 25133 silver badges66 bronze badges add a comment  |  The Overflow Blog A deeper dive into our May 2019 security incident Podcast 307: Owning the code, from integration to delivery Featured on Meta Opt-in alpha test for a new Stacks editor Linked 8 Bash script that shows changing real time values from commands 3 bash scripting and trying to understand an echo command 3 Is it possible to display a running timer in a shell script while copying a directory? Related 1 Parse two files input in for/while loop 30 Is there something wrong with my script or is Bash much slower than Python? 8 Run while true in systemd script 0 kill background while loop 0 how to use a two word password on a script variable? 3 calling a command from script and maintain styling 2 How do I wait for nohup jobs to finish within a shell script? 5 How to cancel a command inside a script without exiting the script itself? Hot Network Questions Finding a proper adverb to end a sentence meaning unnecessary but not otherwise a problem Can the Wish spell change my spell list? Is mirror test a good way to explore alien inhabited world safely? Did Gaiman and Pratchett troll an interviewer who thought they were religious fanatics? Put Option shown as negative value What happened to the Cross Particles stolen by Mitch Carson? Is it a good thing as a teacher to declare things like "Good! Things are becoming clearer already." Inserting © (copyright symbol) using Microsoft Word OpenStreetMap baselayer of just one country Earth is accelerated out of the solar system - do we keep the Moon? Are all atoms spherically symmetric? If so, why are atoms with half-filled/filled sub-shells often quoted as 'especially' spherically symmetric? How are probabilities written in German? I need drivers for Linux install, on my old laptop, Because my laptop is old, will there be any problem if I install Linux? How was I able to access the 14th positional parameter using $14 in a shell script? How Can Vampires Obtain Donors? Is the Wi-Fi in high-speed trains in China reliable and fast enough for audio or video conferences? Construct a permutation tree plot Why don't video conferencing web applications ask permission for screen sharing? My advisor has literally no idea what my research is about and I am freaking out (phd student) How does the federal mask mandate end? Help understanding staccato notation Where were mathematical/science works posted before the arxiv website? Could double jeopardy protect a murderer who bribed the judge and jury to be declared not guilty? Why isn't SpaceX's Starship trial and error great and unique development strategy an opensource project? more hot questions Question feed Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader. lang-bsh Unix & Linux Stack Exchange works best with JavaScript enabled