Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

"I am your density." -- George McFly in "Back to the Future"


devel / comp.unix.shell / Re: Extract the values belong to a specific key.

SubjectAuthor
* Extract the values belong to a specific key.hongy...@gmail.com
+- Re: Extract the values belong to a specific key.Ed Morton
+* Re: Extract the values belong to a specific key.Janis Papanagnou
|`- Re: Extract the values belong to a specific key.Keith Thompson
+- Re: Extract the values belong to a specific key.Grant Taylor
`* Re: Extract the values belong to a specific key.Tavis Ormandy
 `* Re: Extract the values belong to a specific key.hongy...@gmail.com
  +- Re: Extract the values belong to a specific key.hongy...@gmail.com
  +- Re: Extract the values belong to a specific key.Keith Thompson
  `* Re: Extract the values belong to a specific key.Ed Morton
   `- Re: Extract the values belong to a specific key.hongy...@gmail.com

1
Extract the values belong to a specific key.

<d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4700&group=comp.unix.shell#4700

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:a05:620a:4107:: with SMTP id j7mr20624529qko.645.1639147863857;
Fri, 10 Dec 2021 06:51:03 -0800 (PST)
X-Received: by 2002:ad4:5bc4:: with SMTP id t4mr24652968qvt.3.1639147863683;
Fri, 10 Dec 2021 06:51:03 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Fri, 10 Dec 2021 06:51:03 -0800 (PST)
Injection-Info: google-groups.googlegroups.com; posting-host=172.104.65.59; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 172.104.65.59
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
Subject: Extract the values belong to a specific key.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Fri, 10 Dec 2021 14:51:03 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 22
 by: hongy...@gmail.com - Fri, 10 Dec 2021 14:51 UTC

I've a file which has the following content:

"key1":[
"val11",
"val12",
...
],
"key2":[
"val21",
"val22",
...
],
"key3":[
"val31",
"val32",
...
],
...

I want to extract the values belong to a specific key. Any hints?

Regards,
HZ

Re: Extract the values belong to a specific key.

<sovq8l$o9b$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4701&group=comp.unix.shell#4701

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: mortons...@gmail.com (Ed Morton)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 09:03:49 -0600
Organization: A noiseless patient Spider
Lines: 30
Message-ID: <sovq8l$o9b$1@dont-email.me>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 10 Dec 2021 15:03:49 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="ada5a6327ea48e3e1dd667cb1df81b00";
logging-data="24875"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18ms1crwiovSoTnBMJM/Fk1"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.3.2
Cancel-Lock: sha1:a6NBRsElx36IHVCMtC0SOs6jBFU=
In-Reply-To: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
X-Antivirus-Status: Clean
Content-Language: en-US
X-Antivirus: Avast (VPS 211210-0, 12/9/2021), Outbound message
 by: Ed Morton - Fri, 10 Dec 2021 15:03 UTC

On 12/10/2021 8:51 AM, hongy...@gmail.com wrote:
> I've a file which has the following content:
>
> "key1":[
> "val11",
> "val12",
> ...
> ],
> "key2":[
> "val21",
> "val22",
> ...
> ],
> "key3":[
> "val31",
> "val32",
> ...
> ],
> ...
>
> I want to extract the values belong to a specific key. Any hints?
>
> Regards,
> HZ
>

Determine what language that is then google for a tool that understands
whatever language that is and then use that tool to operate on the file.

Ed.

Re: Extract the values belong to a specific key.

<sovqrk$vav$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4702&group=comp.unix.shell#4702

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: janis_pa...@hotmail.com (Janis Papanagnou)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 16:13:56 +0100
Organization: A noiseless patient Spider
Lines: 49
Message-ID: <sovqrk$vav$1@dont-email.me>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 10 Dec 2021 15:13:56 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="668dce94f36490f5df68426bd50ef521";
logging-data="32095"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19U3+WX4KcAHyMb4lwBiehg"
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
Thunderbird/45.8.0
Cancel-Lock: sha1:E1T9qVX0Lkr+3E24wkc/wf4njKY=
In-Reply-To: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
X-Enigmail-Draft-Status: N1110
 by: Janis Papanagnou - Fri, 10 Dec 2021 15:13 UTC

On 10.12.2021 15:51, hongy...@gmail.com wrote:
> I've a file which has the following content:
>
> "key1":[
> "val11",
> "val12",
> ...
> ],
> "key2":[
> "val21",
> "val22",
> ...
> ],
> "key3":[
> "val31",
> "val32",
> ...
> ],
> ...
>
> I want to extract the values belong to a specific key. Any hints?

To get the whole key-block use (for example for "key2")

awk '/"key2":/,/],/'

would produce

"key2":[
"val21",
"val22",
...
],

Adjustments may be necessary if you want something different.
May fail depending on actual data.

Awk is flexible enough to, e.g., omit the first and last line of the
key block, or extract the valXY substrings from the data lines.

Left as homework.

Janis

>
> Regards,
> HZ
>

Re: Extract the values belong to a specific key.

<sp02lb$de0$1@tncsrv09.home.tnetconsulting.net>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4704&group=comp.unix.shell#4704

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!tncsrv06.tnetconsulting.net!tncsrv09.home.tnetconsulting.net!.POSTED.alpha.home.tnetconsulting.net!not-for-mail
From: gtay...@tnetconsulting.net (Grant Taylor)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 10:26:33 -0700
Organization: TNet Consulting
Message-ID: <sp02lb$de0$1@tncsrv09.home.tnetconsulting.net>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Fri, 10 Dec 2021 17:27:07 -0000 (UTC)
Injection-Info: tncsrv09.home.tnetconsulting.net; posting-host="alpha.home.tnetconsulting.net:198.18.18.251";
logging-data="13760"; mail-complaints-to="newsmaster@tnetconsulting.net"
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
Thunderbird/78.13.0
In-Reply-To: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
Content-Language: en-US
 by: Grant Taylor - Fri, 10 Dec 2021 17:26 UTC

On 12/10/21 7:51 AM, hongy...@gmail.com wrote:
> I've a file which has the following content:

Been there.
Done that.
Will do it again.

> I want to extract the values belong to a specific key. Any hints?

If I don't need to do anything other than extract the section, I'd
probably use (GNU) sed.

sed -n '/"key#":/,/\],/p'

(From memory, untested.)

You may need to tweak the pattern matches, especially if the closing
"]," matches multiple parts of the structure. In that case, bound it,
"^ ],$" type thing.

This is probably similar to Janis's awk recommendation. I find sed to
be somewhat simpler (at least in my head) for this than awk.

There may also be tools for working with this format directly. It's
reminiscent of JSON and YAML, both of which have dedicated tools to make
this a lot ... something ... let's go with precise.

--
Grant. . . .
unix || die

Re: Extract the values belong to a specific key.

<87sfv09sz6.fsf@nosuchdomain.example.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4705&group=comp.unix.shell#4705

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: Keith.S....@gmail.com (Keith Thompson)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 11:10:05 -0800
Organization: None to speak of
Lines: 55
Message-ID: <87sfv09sz6.fsf@nosuchdomain.example.com>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
<sovqrk$vav$1@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain
Injection-Info: reader02.eternal-september.org; posting-host="b669a5611549520704c9fe5a579bad30";
logging-data="24466"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/iOOwWf7bceFs1XEElOwOU"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
Cancel-Lock: sha1:gV3BxCIBay96z8NmynstvXiS0Sk=
sha1:XTK1MEH6W4x/Ime2a/73ktz9JRw=
 by: Keith Thompson - Fri, 10 Dec 2021 19:10 UTC

Janis Papanagnou <janis_papanagnou@hotmail.com> writes:
> On 10.12.2021 15:51, hongy...@gmail.com wrote:
>> I've a file which has the following content:
>>
>> "key1":[
>> "val11",
>> "val12",
>> ...
>> ],
>> "key2":[
>> "val21",
>> "val22",
>> ...
>> ],
>> "key3":[
>> "val31",
>> "val32",
>> ...
>> ],
>> ...
>>
>> I want to extract the values belong to a specific key. Any hints?
>
> To get the whole key-block use (for example for "key2")
>
> awk '/"key2":/,/],/'
>
> would produce
>
> "key2":[
> "val21",
> "val22",
> ...
> ],
>
> Adjustments may be necessary if you want something different.
> May fail depending on actual data.
>
> Awk is flexible enough to, e.g., omit the first and last line of the
> key block, or extract the valXY substrings from the data lines.
>
> Left as homework.

Both sed and awk are line-oriented. The file looks very much like JSON.
If that's what it is, then another file might be valid JSON but with a
completely different line structure.

We don't have enough information to suggest a robust solution. I
suspect the real answer is "find a tool that handles JSON", but we don't
know for sure that it's JSON.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips
void Void(void) { Void(); } /* The recursive call of the void */

Re: Extract the values belong to a specific key.

<j1i3gcFcf7nU1@mid.individual.net>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4707&group=comp.unix.shell#4707

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: tav...@gmail.com (Tavis Ormandy)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: 10 Dec 2021 22:43:24 GMT
Lines: 22
Message-ID: <j1i3gcFcf7nU1@mid.individual.net>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
X-Trace: individual.net lcz4vLRHrplg3Ip7vE7I6gioIBAdni3tOepxIiDl75+LVrXDAi
Cancel-Lock: sha1:XyE3Ij6lmuS2Dx6PkdJVrL89c4U=
User-Agent: slrn/pre1.0.4-5 (Linux)
 by: Tavis Ormandy - Fri, 10 Dec 2021 22:43 UTC

On 2021-12-10, hongy...@gmail.com wrote:
> I've a file which has the following content:
>
> "key1":[
> "val11",
> "val12",
> ...
> ],
>
> I want to extract the values belong to a specific key. Any hints?
>

This looks like JSON, there's a tool called jq that makes querying
it easy. Try something like `jq -r '.key2[]' < yourfile`.

Tavis.

--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger taviso@sdf.org
_\_V _( ) _( ) @taviso

Re: Extract the values belong to a specific key.

<8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4708&group=comp.unix.shell#4708

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:a05:620a:1988:: with SMTP id bm8mr22811284qkb.494.1639180339240;
Fri, 10 Dec 2021 15:52:19 -0800 (PST)
X-Received: by 2002:a05:620a:288c:: with SMTP id j12mr23978705qkp.66.1639180339064;
Fri, 10 Dec 2021 15:52:19 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Fri, 10 Dec 2021 15:52:18 -0800 (PST)
In-Reply-To: <j1i3gcFcf7nU1@mid.individual.net>
Injection-Info: google-groups.googlegroups.com; posting-host=139.162.127.102; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 139.162.127.102
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com> <j1i3gcFcf7nU1@mid.individual.net>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
Subject: Re: Extract the values belong to a specific key.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Fri, 10 Dec 2021 23:52:19 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 104
 by: hongy...@gmail.com - Fri, 10 Dec 2021 23:52 UTC

On Saturday, December 11, 2021 at 6:43:30 AM UTC+8, Tavis Ormandy wrote:
> On 2021-12-10, hongy...@gmail.com wrote:
> > I've a file which has the following content:
> >
> > "key1":[
> > "val11",
> > "val12",
> > ...
> > ],
> >
> > I want to extract the values belong to a specific key. Any hints?
> >
> This looks like JSON, there's a tool called jq that makes querying
> it easy. Try something like `jq -r '.key2[]' < yourfile`.

In fact, I am processing the "live/packages_choice.json" file coming from the following image:
http://cdimage.deepin.com/releases/20.3/deepin-desktop-community-20.3-amd64.iso
The original file, although named as a json file, lacks matching `{` at the very beginning, which makes the jq tool fails. If I add the missing `{` to the first line of the file, then jq will work, say, for `dde` key:

$ jq -r '.dde[]' < packages_choice.json
deepin-desktop-server
deepin-default-settings
dde-desktop
dde-dock
dde-launcher
dde-control-center
startdde
dde-session-ui
deepin-artwork
dde-file-manager
dde-qt5integration
plymouth-theme-deepin-logo
deepin-wallpapers
fonts-noto
dde-introduction
dde-kwin
deepin-screensaver
dde
dde-calendar
network-manager-integration-plugins
deepin-terminal

But without adding the missing `{`, the following error will be triggered:

$ jq -r '.dde[]' < /mnt/live/packages_choice.json
jq: error (at <stdin>:2): Cannot index string with string "dde"
parse error: Expected string key before ':' at line 2, column 10

However, both sed and awk can use this poorly formatted json file to complete the following task:

$ awk '/"dde":/,/\]/' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
deepin-desktop-server
deepin-default-settings
dde-desktop
dde-dock
dde-launcher
dde-control-center
startdde
dde-session-ui
deepin-artwork
dde-file-manager
dde-qt5integration
plymouth-theme-deepin-logo
deepin-wallpapers
fonts-noto
dde-introduction
dde-kwin
deepin-screensaver
dde
dde-calendar
network-manager-integration-plugins
deepin-terminal

$ sed -n '/"dde":/,/\]/p' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
deepin-desktop-server
deepin-default-settings
dde-desktop
dde-dock
dde-launcher
dde-control-center
startdde
dde-session-ui
deepin-artwork
dde-file-manager
dde-qt5integration
plymouth-theme-deepin-logo
deepin-wallpapers
fonts-noto
dde-introduction
dde-kwin
deepin-screensaver
dde
dde-calendar
network-manager-integration-plugins
deepin-terminal

Regards,
HZ

Re: Extract the values belong to a specific key.

<0109618f-abf8-4ba9-abe9-8859c48f81cbn@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4709&group=comp.unix.shell#4709

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:ac8:5848:: with SMTP id h8mr32278858qth.488.1639182203414;
Fri, 10 Dec 2021 16:23:23 -0800 (PST)
X-Received: by 2002:a05:622a:189:: with SMTP id s9mr30194420qtw.352.1639182203046;
Fri, 10 Dec 2021 16:23:23 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Fri, 10 Dec 2021 16:23:22 -0800 (PST)
In-Reply-To: <8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=206.189.82.58; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 206.189.82.58
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
<j1i3gcFcf7nU1@mid.individual.net> <8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <0109618f-abf8-4ba9-abe9-8859c48f81cbn@googlegroups.com>
Subject: Re: Extract the values belong to a specific key.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Sat, 11 Dec 2021 00:23:23 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 129
 by: hongy...@gmail.com - Sat, 11 Dec 2021 00:23 UTC

On Saturday, December 11, 2021 at 7:52:21 AM UTC+8, hongy...@gmail.com wrote:
> On Saturday, December 11, 2021 at 6:43:30 AM UTC+8, Tavis Ormandy wrote:
> > On 2021-12-10, hongy...@gmail.com wrote:
> > > I've a file which has the following content:
> > >
> > > "key1":[
> > > "val11",
> > > "val12",
> > > ...
> > > ],
> > >
> > > I want to extract the values belong to a specific key. Any hints?
> > >
> > This looks like JSON, there's a tool called jq that makes querying
> > it easy. Try something like `jq -r '.key2[]' < yourfile`.
> In fact, I am processing the "live/packages_choice.json" file coming from the following image:
> http://cdimage.deepin.com/releases/20.3/deepin-desktop-community-20.3-amd64.iso
>
> The original file, although named as a json file, lacks matching `{` at the very beginning, which makes the jq tool fails. If I add the missing `{` to the first line of the file, then jq will work, say, for `dde` key:
>
> $ jq -r '.dde[]' < packages_choice.json
> deepin-desktop-server
> deepin-default-settings
> dde-desktop
> dde-dock
> dde-launcher
> dde-control-center
> startdde
> dde-session-ui
> deepin-artwork
> dde-file-manager
> dde-qt5integration
> plymouth-theme-deepin-logo
> deepin-wallpapers
> fonts-noto
> dde-introduction
> dde-kwin
> deepin-screensaver
> dde
> dde-calendar
> network-manager-integration-plugins
> deepin-terminal
>
> But without adding the missing `{`, the following error will be triggered:
>
> $ jq -r '.dde[]' < /mnt/live/packages_choice.json
> jq: error (at <stdin>:2): Cannot index string with string "dde"
> parse error: Expected string key before ':' at line 2, column 10
>
>
> However, both sed and awk can use this poorly formatted json file to complete the following task:
>
> $ awk '/"dde":/,/\]/' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
> deepin-desktop-server
> deepin-default-settings
> dde-desktop
> dde-dock
> dde-launcher
> dde-control-center
> startdde
> dde-session-ui
> deepin-artwork
> dde-file-manager
> dde-qt5integration
> plymouth-theme-deepin-logo
> deepin-wallpapers
> fonts-noto
> dde-introduction
> dde-kwin
> deepin-screensaver
> dde
> dde-calendar
> network-manager-integration-plugins
> deepin-terminal
>
> $ sed -n '/"dde":/,/\]/p' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
> deepin-desktop-server
> deepin-default-settings
> dde-desktop
> dde-dock
> dde-launcher
> dde-control-center
> startdde
> dde-session-ui
> deepin-artwork
> dde-file-manager
> dde-qt5integration
> plymouth-theme-deepin-logo
> deepin-wallpapers
> fonts-noto
> dde-introduction
> dde-kwin
> deepin-screensaver
> dde
> dde-calendar
> network-manager-integration-plugins
> deepin-terminal

In fact, the final format I want should look like this:

$ sudo mount deepin-desktop-community-20.3-amd64.iso /mnt
$ sed -n '/"dde":/,/\]/p' /mnt/live/packages_choice.json |
sed '1d;$d' |
awk -F\" '{print $2}' |
egrep -v '^(dde|plymouth-.*|dde-session-.*|network-manager-.*)$' |
awk '{line[NR]=$0} END {for (i=1;i<=length(line);i++) {if (i == 1 ) {print "env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends "line[i]" \\"} else { if (i == length(line)) { print line[i]} else { print line[i]" \\"}}}}'
env DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends deepin-desktop-server \
deepin-default-settings \
dde-desktop \
dde-dock \
dde-launcher \
dde-control-center \
startdde \
deepin-artwork \
dde-file-manager \
dde-qt5integration \
deepin-wallpapers \
fonts-noto \
dde-introduction \
dde-kwin \
deepin-screensaver \
dde-calendar \
deepin-terminal

Although the script code written by me above look very ugly and crappy, they do represent the style of the data I ultimately need to extract and obtain. It's currently used by the Dockerfile here [1].

[1] https://github.com/hongyi-zhao/dockerfile

HZ

Re: Extract the values belong to a specific key.

<87o85o9dty.fsf@nosuchdomain.example.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4710&group=comp.unix.shell#4710

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: Keith.S....@gmail.com (Keith Thompson)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 16:37:13 -0800
Organization: None to speak of
Lines: 71
Message-ID: <87o85o9dty.fsf@nosuchdomain.example.com>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
<j1i3gcFcf7nU1@mid.individual.net>
<8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain
Injection-Info: reader02.eternal-september.org; posting-host="f9909e9f207a8935fdc9a42a49be8853";
logging-data="3034"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19vG5pdZTmF+8YIIVbIytmg"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
Cancel-Lock: sha1:DLlTIRtn3lbih7Mq2vUVllxKonE=
sha1:UcE1xC8zyuTL97RIpCJpbQjGQio=
 by: Keith Thompson - Sat, 11 Dec 2021 00:37 UTC

"hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
> On Saturday, December 11, 2021 at 6:43:30 AM UTC+8, Tavis Ormandy wrote:
>> On 2021-12-10, hongy...@gmail.com wrote:
>> > I've a file which has the following content:
>> >
>> > "key1":[
>> > "val11",
>> > "val12",
>> > ...
>> > ],
>> >
>> > I want to extract the values belong to a specific key. Any hints?
>> >
>> This looks like JSON, there's a tool called jq that makes querying
>> it easy. Try something like `jq -r '.key2[]' < yourfile`.
>
> In fact, I am processing the "live/packages_choice.json" file coming from the following image:
> http://cdimage.deepin.com/releases/20.3/deepin-desktop-community-20.3-amd64.iso

(Deepin is apparently a Chinese Linux distribution.)

If you had mentioned from the start that file's name ends in ".json", it
would have saved a lot of time. Don't hide information like that.

> The original file, although named as a json file, lacks matching `{`
> at the very beginning, which makes the jq tool fails. If I add the
> missing `{` to the first line of the file, then jq will work, say, for
> `dde` key:

So it's not valid JSON, while the file name implies that it is.
Perhaps consider reporting this problem to the site that provided it.
(Maybe there's some valid reason for it, though.)

> $ jq -r '.dde[]' < packages_choice.json
[...]
>
> But without adding the missing `{`, the following error will be triggered:
>
> $ jq -r '.dde[]' < /mnt/live/packages_choice.json
> jq: error (at <stdin>:2): Cannot index string with string "dde"
> parse error: Expected string key before ':' at line 2, column 10

I'd say the best way to read information from the file is to create a
copy with the required '{' and '}' added and then use jq. Or just
something like:

( echo '{' ; cat packages_choice.json ; echo '}' ) | jq ...

You've already made jq work.

If your only requirement is to parse that one specific version of that
one specific file, do whatever works. If you might need to extract
information from a different or future version, any sed/awk solution is
likely to break (since line breaks can mostly be inserted or removed
arbitrarily in JSON).

> However, both sed and awk can use this poorly formatted json file to
> complete the following task:
>
> $ awk '/"dde":/,/\]/' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
[snip]
> $ sed -n '/"dde":/,/\]/p' /mnt/live/packages_choice.json | sed '1d;$d' | awk -F\" '{print $2}'
[snip]

You could almost certainly do that with a single awk command. But
again, awk and sed are probably the wrong tools.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips
void Void(void) { Void(); } /* The recursive call of the void */

Re: Extract the values belong to a specific key.

<sp0s7v$am3$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4711&group=comp.unix.shell#4711

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!eternal-september.org!reader02.eternal-september.org!.POSTED!not-for-mail
From: mortons...@gmail.com (Ed Morton)
Newsgroups: comp.unix.shell
Subject: Re: Extract the values belong to a specific key.
Date: Fri, 10 Dec 2021 18:43:42 -0600
Organization: A noiseless patient Spider
Lines: 32
Message-ID: <sp0s7v$am3$1@dont-email.me>
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
<j1i3gcFcf7nU1@mid.individual.net>
<8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sat, 11 Dec 2021 00:43:43 -0000 (UTC)
Injection-Info: reader02.eternal-september.org; posting-host="2cc87d5e0009bb53d5dc06f86596d185";
logging-data="10947"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/DazUC/u+Gs4lAM5x6s6IB"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.3.2
Cancel-Lock: sha1:W7aHjTzLnpw4kq8uerIfOCzn+lU=
In-Reply-To: <8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
X-Antivirus-Status: Clean
Content-Language: en-US
X-Antivirus: Avast (VPS 211210-4, 12/10/2021), Outbound message
 by: Ed Morton - Sat, 11 Dec 2021 00:43 UTC

On 12/10/2021 5:52 PM, hongy...@gmail.com wrote:
> On Saturday, December 11, 2021 at 6:43:30 AM UTC+8, Tavis Ormandy wrote:
>> On 2021-12-10, hongy...@gmail.com wrote:
>>> I've a file which has the following content:
>>>
>>> "key1":[
>>> "val11",
>>> "val12",
>>> ...
>>> ],
>>>
>>> I want to extract the values belong to a specific key. Any hints?
>>>
>> This looks like JSON, there's a tool called jq that makes querying
>> it easy. Try something like `jq -r '.key2[]' < yourfile`.
>
> In fact, I am processing the "live/packages_choice.json" file coming from the following image:
> http://cdimage.deepin.com/releases/20.3/deepin-desktop-community-20.3-amd64.iso
>
> The original file, although named as a json file, lacks matching `{` at the very beginning, which makes the jq tool fails. If I add the missing `{` to the first line of the file, then jq will work, say, for `dde` key:
>
> $ jq -r '.dde[]' < packages_choice.json

Then why would you look for anything beyond:

{ echo '{'; cat packages_choice.json; } | jq -r '.dde[]'

or similar? Anything you come up with using sed or awk to parse the JSON
will be far less robust than using `jq` and it doesn't seem necessary if
all you need to do is add a leading `{` to the input and then `jq` works.

Ed.

Re: Extract the values belong to a specific key.

<b793fb8d-900c-4099-8d62-4a9e88e2b2b9n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4712&group=comp.unix.shell#4712

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:ad4:5de3:: with SMTP id jn3mr29729713qvb.22.1639184185775;
Fri, 10 Dec 2021 16:56:25 -0800 (PST)
X-Received: by 2002:a05:620a:1a9d:: with SMTP id bl29mr23750187qkb.55.1639184185604;
Fri, 10 Dec 2021 16:56:25 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Fri, 10 Dec 2021 16:56:25 -0800 (PST)
In-Reply-To: <sp0s7v$am3$1@dont-email.me>
Injection-Info: google-groups.googlegroups.com; posting-host=139.59.109.137; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 139.59.109.137
References: <d559bd27-5dda-46d9-b853-471bae077d66n@googlegroups.com>
<j1i3gcFcf7nU1@mid.individual.net> <8a9d3908-d26e-4473-bd4f-8f4a2284dc6en@googlegroups.com>
<sp0s7v$am3$1@dont-email.me>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <b793fb8d-900c-4099-8d62-4a9e88e2b2b9n@googlegroups.com>
Subject: Re: Extract the values belong to a specific key.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Sat, 11 Dec 2021 00:56:25 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 58
 by: hongy...@gmail.com - Sat, 11 Dec 2021 00:56 UTC

On Saturday, December 11, 2021 at 8:43:48 AM UTC+8, Ed Morton wrote:
> On 12/10/2021 5:52 PM, hongy...@gmail.com wrote:
> > On Saturday, December 11, 2021 at 6:43:30 AM UTC+8, Tavis Ormandy wrote:
> >> On 2021-12-10, hongy...@gmail.com wrote:
> >>> I've a file which has the following content:
> >>>
> >>> "key1":[
> >>> "val11",
> >>> "val12",
> >>> ...
> >>> ],
> >>>
> >>> I want to extract the values belong to a specific key. Any hints?
> >>>
> >> This looks like JSON, there's a tool called jq that makes querying
> >> it easy. Try something like `jq -r '.key2[]' < yourfile`.
> >
> > In fact, I am processing the "live/packages_choice.json" file coming from the following image:
> > http://cdimage.deepin.com/releases/20.3/deepin-desktop-community-20.3-amd64.iso
> >
> > The original file, although named as a json file, lacks matching `{` at the very beginning, which makes the jq tool fails. If I add the missing `{` to the first line of the file, then jq will work, say, for `dde` key:
> >
> > $ jq -r '.dde[]' < packages_choice.json
> Then why would you look for anything beyond:
>
> { echo '{'; cat packages_choice.json; } | jq -r '.dde[]'

Yes. It works:

$ { echo '{'; cat /mnt/live/packages_choice.json; } | jq -r '.dde[]'
deepin-desktop-server
deepin-default-settings
dde-desktop
dde-dock
dde-launcher
dde-control-center
startdde
dde-session-ui
deepin-artwork
dde-file-manager
dde-qt5integration
plymouth-theme-deepin-logo
deepin-wallpapers
fonts-noto
dde-introduction
dde-kwin
deepin-screensaver
dde
dde-calendar
network-manager-integration-plugins
deepin-terminal

> or similar? Anything you come up with using sed or awk to parse the JSON
> will be far less robust than using `jq` and it doesn't seem necessary if
> all you need to do is add a leading `{` to the input and then `jq` works.

It was only after I checked the file that I discovered the problem. This is a bug/error introduced by the image creator and should be fixed in a future version. So, I agree that in this case, `jq` is the right tool.
HZ

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor