Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

An elephant is a mouse with an operating system.


devel / comp.unix.shell / curl 'https://i.jpillora.com/chisel!' vs open it in the browser.

SubjectAuthor
* curl 'https://i.jpillora.com/chisel!' vs open it in the browser.hongy...@gmail.com
`* Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.Russell Marks
 `- Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.hongy...@gmail.com

1
curl 'https://i.jpillora.com/chisel!' vs open it in the browser.

<4f1d19c8-300b-4c24-9c29-9d67e28b2bc1n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4556&group=comp.unix.shell#4556

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:a0c:8c81:: with SMTP id p1mr15644911qvb.7.1635581828070;
Sat, 30 Oct 2021 01:17:08 -0700 (PDT)
X-Received: by 2002:a05:620a:448e:: with SMTP id x14mr13098828qkp.459.1635581827902;
Sat, 30 Oct 2021 01:17:07 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Sat, 30 Oct 2021 01:17:07 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=128.199.66.196; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 128.199.66.196
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <4f1d19c8-300b-4c24-9c29-9d67e28b2bc1n@googlegroups.com>
Subject: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Sat, 30 Oct 2021 08:17:08 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 178
 by: hongy...@gmail.com - Sat, 30 Oct 2021 08:17 UTC

Chisel's website describes the following installation method [1]:

See the latest release or download and install it now with curl https://i.jpillora.com/chisel! | bash

[1] https://github.com/jpillora/chisel#binaries

But I tried to provide the above URL to curl and open it in Firefox. To my surprise, it gave different results:

Testing with curl:

##########################
$ curl https://i.jpillora.com/chisel
#!/bin/bash
TMP_DIR="/tmp/tmpinstalldir"
function cleanup {
echo rm -rf $TMP_DIR > /dev/null
} function fail {
cleanup
msg=$1
echo "============"
echo "Error: $msg" 1>&2
exit 1
} function install {
#settings
USER="jpillora"
PROG="chisel"
MOVE="false"
RELEASE="v1.7.6"
INSECURE="false"
OUT_DIR="$(pwd)"
GH="https://github.com"
#bash check
[ ! "$BASH_VERSION" ] && fail "Please use bash instead"
[ ! -d $OUT_DIR ] && fail "output directory missing: $OUT_DIR"
#dependency check, assume we are a standard POISX machine
which find > /dev/null || fail "find not installed"
which xargs > /dev/null || fail "xargs not installed"
which sort > /dev/null || fail "sort not installed"
which tail > /dev/null || fail "tail not installed"
which cut > /dev/null || fail "cut not installed"
which du > /dev/null || fail "du not installed"
GET=""
if which curl > /dev/null; then
GET="curl"
if [[ $INSECURE = "true" ]]; then GET="$GET --insecure"; fi
GET="$GET --fail -# -L"
elif which wget > /dev/null; then
GET="wget"
if [[ $INSECURE = "true" ]]; then GET="$GET --no-check-certificate"; fi
GET="$GET -qO-"
else
fail "neither wget/curl are installed"
fi
#find OS #TODO BSDs and other posixs
case `uname -s` in
Darwin) OS="darwin";;
Linux) OS="linux";;
*) fail "unknown os: $(uname -s)";;
esac
#find ARCH
if uname -m | grep 64 > /dev/null; then
ARCH="amd64"
elif uname -m | grep arm > /dev/null; then
ARCH="arm" #TODO armv6/v7
elif uname -m | grep 386 > /dev/null; then
ARCH="386"
else
fail "unknown arch: $(uname -m)"
fi
#choose from asset list
URL=""
FTYPE=""
case "${OS}_${ARCH}" in
"darwin_amd64")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz"
FTYPE=".gz"
;;
"darwin_arm")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz"
FTYPE=".gz"
;;
"linux_386")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz"
FTYPE=".gz"
;;
"linux_amd64")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz"
FTYPE=".gz"
;;
"linux_arm")
URL="https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz"
FTYPE=".gz"
;;
*) fail "No asset for platform ${OS}-${ARCH}";;
esac
#got URL! download it...
echo -n "Downloading $USER/$PROG $RELEASE"
echo "....."
#enter tempdir
mkdir -p $TMP_DIR
cd $TMP_DIR
if [[ $FTYPE = ".gz" ]]; then
which gzip > /dev/null || fail "gzip is not installed"
#gzipped binary
NAME="${PROG}_${OS}_${ARCH}.gz"
GZURL="$GH/releases/download/$RELEASE/$NAME"
#gz download!
bash -c "$GET $URL" | gzip -d - > $PROG || fail "download failed"
elif [[ $FTYPE = ".tar.gz" ]] || [[ $FTYPE = ".tgz" ]]; then
#check if archiver progs installed
which tar > /dev/null || fail "tar is not installed"
which gzip > /dev/null || fail "gzip is not installed"
bash -c "$GET $URL" | tar zxf - || fail "download failed"
elif [[ $FTYPE = ".zip" ]]; then
which unzip > /dev/null || fail "unzip is not installed"
bash -c "$GET $URL" > tmp.zip || fail "download failed"
unzip -o -qq tmp.zip || fail "unzip failed"
rm tmp.zip || fail "cleanup failed"
elif [[ $FTYPE = "" ]]; then
bash -c "$GET $URL" > "chisel_${OS}_${ARCH}" || fail "download failed"
else
fail "unknown file type: $FTYPE"
fi
#search subtree largest file (bin)
TMP_BIN=$(find . -type f | xargs du | sort -n | tail -n 1 | cut -f 2)
if [ ! -f "$TMP_BIN" ]; then
fail "could not find find binary (largest file)"
fi
#ensure its larger than 1MB
if [[ $(du -m $TMP_BIN | cut -f1) -lt 1 ]]; then
fail "no binary found ($TMP_BIN is not larger than 1MB)"
fi
#move into PATH or cwd
chmod +x $TMP_BIN || fail "chmod +x failed"
mv $TMP_BIN $OUT_DIR/$PROG || fail "mv failed" #FINAL STEP!
echo "Downloaded to $OUT_DIR/$PROG"
#done
cleanup
} install
##########################

Open it in Firefox:

########################
repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6
release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer
########################

OTOH, I also tried with the following url, `https://i.jpillora.com/chisel', i.e., with the last `!' character stripped, and it still gives the same results.

Any hints for the above phenomenon?

Regards,
HZ

Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.

<669fJ.120131$Ah33.80263@usenetxs.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4558&group=comp.unix.shell#4558

 copy link   Newsgroups: comp.unix.shell
Path: i2pn2.org!i2pn.org!paganini.bofh.team!news.dns-netz.com!news.freedyn.net!newsreader4.netcologne.de!news.netcologne.de!peer03.ams1!peer.ams1.xlned.com!news.xlned.com!peer02.ams4!peer.am4.highwinds-media.com!news.highwinds-media.com!fx04.ams4.POSTED!not-for-mail
From: zgedn...@spam^H^H^H^Hgmail.com (Russell Marks)
Newsgroups: comp.unix.shell
Subject: Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.
References: <4f1d19c8-300b-4c24-9c29-9d67e28b2bc1n@googlegroups.com>
Organization: this space intentionally left blank
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Lines: 47
Message-ID: <669fJ.120131$Ah33.80263@usenetxs.com>
X-Complaints-To: https://www.astraweb.com/aup
NNTP-Posting-Date: Sat, 30 Oct 2021 10:27:14 UTC
Date: Sat, 30 Oct 2021 10:27:14 GMT
X-Received-Bytes: 2224
 by: Russell Marks - Sat, 30 Oct 2021 10:27 UTC

"hongy...@gmail.com" <hongyi.zhao@gmail.com> wrote:

[longer lines wrapped]
> Chisel's website describes the following installation method [1]:
>
> See the latest release or download and install it now with
> curl https://i.jpillora.com/chisel!" rel="nofollow" target="_blank">https://i.jpillora.com/chisel! | bash
>
> [1] https://github.com/jpillora/chisel#binaries" rel="nofollow" target="_blank">https://github.com/jpillora/chisel#binaries
>
> But I tried to provide the above URL to curl and open it in Firefox.
> To my surprise, it gave different results:
>
> Testing with curl:
>
> ##########################
> $ curl https://i.jpillora.com/chisel
> #!/bin/bash
> TMP_DIR="/tmp/tmpinstalldir"
[...]
> ##########################
>
> Open it in Firefox:
>
> ########################
> repository: https://github.com/jpillora/chisel
> user: jpillora
> program: chisel
> release: v1.7.6
[...]
> ########################
>
> OTOH, I also tried with the following url,
> `https://i.jpillora.com/chisel', i.e., with the last `!' character
> stripped, and it still gives the same results.
>
> Any hints for the above phenomenon?

The differing content is probably based on the User-Agent header,
which the web server will have received before serving the page. A
quick test seems to support this - using wget returns something
similar to the curl version, but "wget -U ''" (which doesn't send a
User-Agent at all) returns the version you got in Firefox. That
version is also returned for "busybox wget" and "lynx -source",
downloading options that presumably weren't accounted for.

-Rus.

Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.

<2badb2a8-d461-4b68-87e5-fb925db430b7n@googlegroups.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=4559&group=comp.unix.shell#4559

 copy link   Newsgroups: comp.unix.shell
X-Received: by 2002:ac8:6112:: with SMTP id a18mr17810318qtm.401.1635595697178;
Sat, 30 Oct 2021 05:08:17 -0700 (PDT)
X-Received: by 2002:a05:622a:191:: with SMTP id s17mr18293779qtw.18.1635595697008;
Sat, 30 Oct 2021 05:08:17 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.unix.shell
Date: Sat, 30 Oct 2021 05:08:16 -0700 (PDT)
In-Reply-To: <669fJ.120131$Ah33.80263@usenetxs.com>
Injection-Info: google-groups.googlegroups.com; posting-host=47.241.171.246; posting-account=kF0ZaAoAAACPbiK5gldhAyX5qTd3krV2
NNTP-Posting-Host: 47.241.171.246
References: <4f1d19c8-300b-4c24-9c29-9d67e28b2bc1n@googlegroups.com> <669fJ.120131$Ah33.80263@usenetxs.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <2badb2a8-d461-4b68-87e5-fb925db430b7n@googlegroups.com>
Subject: Re: curl 'https://i.jpillora.com/chisel!' vs open it in the browser.
From: hongyi.z...@gmail.com (hongy...@gmail.com)
Injection-Date: Sat, 30 Oct 2021 12:08:17 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 94
 by: hongy...@gmail.com - Sat, 30 Oct 2021 12:08 UTC

On Saturday, October 30, 2021 at 6:27:19 PM UTC+8, Russell Marks wrote:
> "hongy...@gmail.com" <hongy...@gmail.com> wrote:
>
> [longer lines wrapped]
> > Chisel's website describes the following installation method [1]:
> >
> > See the latest release or download and install it now with
> > curl https://i.jpillora.com/chisel!" rel="nofollow" target="_blank">https://i.jpillora.com/chisel! | bash
> >
> > [1] https://github.com/jpillora/chisel#binaries" rel="nofollow" target="_blank">https://github.com/jpillora/chisel#binaries
> >
> > But I tried to provide the above URL to curl and open it in Firefox.
> > To my surprise, it gave different results:
> >
> > Testing with curl:
> >
> > ##########################
> > $ curl https://i.jpillora.com/chisel
> > #!/bin/bash
> > TMP_DIR="/tmp/tmpinstalldir"
> [...]
> > ##########################
> >
> > Open it in Firefox:
> >
> > ########################
> > repository: https://github.com/jpillora/chisel
> > user: jpillora
> > program: chisel
> > release: v1.7.6
> [...]
> > ########################
> >
> > OTOH, I also tried with the following url,
> > `https://i.jpillora.com/chisel', i.e., with the last `!' character
> > stripped, and it still gives the same results.
> >
> > Any hints for the above phenomenon?
> The differing content is probably based on the User-Agent header,
> which the web server will have received before serving the page. A
> quick test seems to support this - using wget returns something
> similar to the curl version, but "wget -U ''" (which doesn't send a
> User-Agent at all) returns the version you got in Firefox.

Thank you for pointing this out. The corresponding curl option is "curl -A ''", as shown below:

werner@X10DAi-00:~$ wget -qU '' https://i.jpillora.com/chisel! -O-
repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6
release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer

werner@X10DAi-00:~$ curl -sA '' https://i.jpillora.com/chisel!
repository: https://github.com/jpillora/chisel
user: jpillora
program: chisel
release: v1.7.6
release assets:
[#01] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_amd64.gz
[#02] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_darwin_arm64.gz
[#03] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_386.gz
[#04] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_amd64.gz
[#05] https://github.com/jpillora/chisel/releases/download/v1.7.6/chisel_1.7.6_linux_arm64.gz
move-into-path: true

to see shell script, visit:
i.jpillora.com/chisel!?type=script

for more information on this server, visit:
github.com/jpillora/installer

As I've pointed earlier, use the following URLs are equivalent:

https://i.jpillora.com/chisel!" rel="nofollow" target="_blank">https://i.jpillora.com/chisel!
https://i.jpillora.com/chisel

> That version is also returned for "busybox wget" and "lynx -source",
> downloading options that presumably weren't accounted for.
>
> -Rus.

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor