Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

"Gravitation cannot be held responsible for people falling in love." -- Albert Einstein


devel / comp.arch.fpga / Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

SubjectAuthor
* Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignChris Adams
`* Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignTheo
 `* Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignChris Adams
  `* Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignTheo
   `* Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignChris Adams
    `* Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignChris Adams
     `- Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 DesignTheo

1
Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=216&group=comp.arch.fpga#216

  copy link   Newsgroups: comp.arch.fpga
X-Received: by 2002:ac8:5f0c:: with SMTP id x12mr13303915qta.309.1635528067070;
Fri, 29 Oct 2021 10:21:07 -0700 (PDT)
X-Received: by 2002:ac8:5bd5:: with SMTP id b21mr13422248qtb.298.1635528066887;
Fri, 29 Oct 2021 10:21:06 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.fpga
Date: Fri, 29 Oct 2021 10:21:06 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=142.76.30.252; posting-account=pcMqhQoAAAATrGFhg20e9bXob643rZ-7
NNTP-Posting-Host: 142.76.30.252
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com>
Subject: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
From: chr...@chrisada.co.uk (Chris Adams)
Injection-Date: Fri, 29 Oct 2021 17:21:07 +0000
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Lines: 30
 by: Chris Adams - Fri, 29 Oct 2021 17:21 UTC

Hello,

I have a Stratix 10 design that is based around an ip core generated using Intel's HLS. The core does some simple floating point operations and by itself uses very few resources (1 DSP, a few hundred flops etc).

This core sits inside a generate statement like this:

generate
for(i = 0; i < SOMEBIGNUMBER; i=i+1)
myhlscore u0 (inputs, outputs);
....

The design works and is proven in simulation and in hardware.

The problem comes when I try to increase the value of SOMEBIGNUMBER. Despite there being adequate resources, using values above 200 or so make the synthesis tool run out of memory.

I cannot alleviate this easily by adding more memory - I already tried synthesizing on a computer with 256GB memory and a 200GB swap space and quartus ate it all up before dying.

I'm using a .ip file from HLS right now. I'm wondering is there is some way to pre-synthesis the module and keep the results, or is there someway I need to write the generate statement so that it caches less? Perhaps there are some synthesis settings I can change?

Thanks,
C

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<aPf*tX2xy@news.chiark.greenend.org.uk>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=217&group=comp.arch.fpga#217

  copy link   Newsgroups: comp.arch.fpga
Path: i2pn2.org!i2pn.org!paganini.bofh.team!newsfeed.xs3.de!callisto.xs3.de!nntp-feed.chiark.greenend.org.uk!ewrotcd!.POSTED!not-for-mail
From: theom+n...@chiark.greenend.org.uk (Theo)
Newsgroups: comp.arch.fpga
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
Date: 30 Oct 2021 21:55:40 +0100 (BST)
Organization: University of Cambridge, England
Lines: 25
Message-ID: <aPf*tX2xy@news.chiark.greenend.org.uk>
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com>
NNTP-Posting-Host: chiark.greenend.org.uk
X-Trace: chiark.greenend.org.uk 1635627342 15846 212.13.197.229 (30 Oct 2021 20:55:42 GMT)
X-Complaints-To: abuse@chiark.greenend.org.uk
NNTP-Posting-Date: Sat, 30 Oct 2021 20:55:42 +0000 (UTC)
User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (Linux/3.16.0-11-amd64 (x86_64))
Originator: theom@chiark.greenend.org.uk ([212.13.197.229])
 by: Theo - Sat, 30 Oct 2021 20:55 UTC

Chris Adams <chris@chrisada.co.uk> wrote:
> I'm using a .ip file from HLS right now. I'm wondering is there is some
> way to pre-synthesis the module and keep the results, or is there someway
> I need to write the generate statement so that it caches less? Perhaps
> there are some synthesis settings I can change?

You can put the module in a design partition. That should mean the tools
can reuse the block again if it's repeated, and won't try and flatten
everything out before routing, which is probably what causes the out of
memory errors.

Design Partition Planner is the tool to do this. I haven't watched it all,
but this video seems to cover how to use partitions:
https://www.youtube.com/watch?v=AW9kev4lM7g

Beware that if you're using a tool that generates Verilog (not sure if HLS
is here) you need to keep the structure through to the Verilog level. In
other words if your myhlscore() is the same Verilog module on each iteration
of the generate loop that's fine but if your HLS core is producing
SOMEBIGNUMBER of different verilog modules, or flattening them to one
enormous module, that's going to be a problem. It looks like your HLS is
constrained to inside the module and your generate is in regular Verilog so
that's probably OK.

Theo

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=225&group=comp.arch.fpga#225

  copy link   Newsgroups: comp.arch.fpga
X-Received: by 2002:a37:8ec6:: with SMTP id q189mr6982441qkd.145.1636476893543;
Tue, 09 Nov 2021 08:54:53 -0800 (PST)
X-Received: by 2002:a05:622a:5c9:: with SMTP id d9mr10252330qtb.361.1636476893377;
Tue, 09 Nov 2021 08:54:53 -0800 (PST)
Path: i2pn2.org!i2pn.org!paganini.bofh.team!news.dns-netz.com!news.freedyn.net!newsreader4.netcologne.de!news.netcologne.de!peer03.ams1!peer.ams1.xlned.com!news.xlned.com!peer01.iad!feed-me.highwinds-media.com!news.highwinds-media.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.fpga
Date: Tue, 9 Nov 2021 08:54:53 -0800 (PST)
In-Reply-To: <aPf*tX2xy@news.chiark.greenend.org.uk>
Injection-Info: google-groups.googlegroups.com; posting-host=142.76.30.252; posting-account=pcMqhQoAAAATrGFhg20e9bXob643rZ-7
NNTP-Posting-Host: 142.76.30.252
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com> <aPf*tX2xy@news.chiark.greenend.org.uk>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com>
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
From: chr...@chrisada.co.uk (Chris Adams)
Injection-Date: Tue, 09 Nov 2021 16:54:53 +0000
Content-Type: text/plain; charset="UTF-8"
X-Received-Bytes: 2746
 by: Chris Adams - Tue, 9 Nov 2021 16:54 UTC

On Saturday, 30 October 2021 at 16:55:47 UTC-4, Theo wrote:
> Chris Adams <> wrote:
> > I'm using a .ip file from HLS right now. I'm wondering is there is some
> > way to pre-synthesis the module and keep the results, or is there someway
> > I need to write the generate statement so that it caches less? Perhaps
> > there are some synthesis settings I can change?
> You can put the module in a design partition. That should mean the tools
> can reuse the block again if it's repeated, and won't try and flatten
> everything out before routing, which is probably what causes the out of
> memory errors.
>
> Design Partition Planner is the tool to do this. I haven't watched it all,
> but this video seems to cover how to use partitions:
> https://www.youtube.com/watch?v=AW9kev4lM7g
>
> Beware that if you're using a tool that generates Verilog (not sure if HLS
> is here) you need to keep the structure through to the Verilog level. In
> other words if your myhlscore() is the same Verilog module on each iteration
> of the generate loop that's fine but if your HLS core is producing
> SOMEBIGNUMBER of different verilog modules, or flattening them to one
> enormous module, that's going to be a problem. It looks like your HLS is
> constrained to inside the module and your generate is in regular Verilog so
> that's probably OK.
>
> Theo
Good idea. We tried using a design partition, but even the elaboration stage uses over 120GB of memory before crashing. I may try setting up an extremely large swap file.

Chris

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<aPf*dzRyy@news.chiark.greenend.org.uk>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=226&group=comp.arch.fpga#226

  copy link   Newsgroups: comp.arch.fpga
Path: i2pn2.org!i2pn.org!aioe.org!nntp.terraraq.uk!nntp-feed.chiark.greenend.org.uk!ewrotcd!.POSTED!not-for-mail
From: theom+n...@chiark.greenend.org.uk (Theo)
Newsgroups: comp.arch.fpga
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
Date: 09 Nov 2021 20:24:44 +0000 (GMT)
Organization: University of Cambridge, England
Lines: 20
Message-ID: <aPf*dzRyy@news.chiark.greenend.org.uk>
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com> <aPf*tX2xy@news.chiark.greenend.org.uk> <fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com>
NNTP-Posting-Host: chiark.greenend.org.uk
X-Trace: chiark.greenend.org.uk 1636489486 31150 212.13.197.229 (9 Nov 2021 20:24:46 GMT)
X-Complaints-To: abuse@chiark.greenend.org.uk
NNTP-Posting-Date: Tue, 9 Nov 2021 20:24:46 +0000 (UTC)
User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (Linux/3.16.0-11-amd64 (x86_64))
Originator: theom@chiark.greenend.org.uk ([212.13.197.229])
 by: Theo - Tue, 9 Nov 2021 20:24 UTC

Chris Adams <chris@chrisada.co.uk> wrote:

> Good idea. We tried using a design partition, but even the elaboration
> stage uses over 120GB of memory before crashing. I may try setting up an
> extremely large swap file.

I wonder if the generate statement is causing the trouble, and whether just
having a pile of flat instantiations would be any different?

One other thing you could try, if the I/O isn't too troublesome, is a tree
of instantiations. module A2 contains two instances of the module A, module
A4 contains two instances of A2, module A8 two of A4, etc. That way it
keeps the complexity at each level of hierarchy down. If the synthesiser is
blowing up at the elaboration stage it could help if the elaboration in a
specific module is within limits. I've not tried it though.

I've had Stratix 10 builds need >16GB but <64GB, for what it's worth, but
these weren't super full designs.

Theo

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<2187e4d3-bb57-4885-9dfc-c68927f12a6fn@googlegroups.com>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=240&group=comp.arch.fpga#240

  copy link   Newsgroups: comp.arch.fpga
X-Received: by 2002:ad4:58cf:: with SMTP id dh15mr5258850qvb.125.1639488903635;
Tue, 14 Dec 2021 05:35:03 -0800 (PST)
X-Received: by 2002:a37:65d2:: with SMTP id z201mr3901684qkb.467.1639488903420;
Tue, 14 Dec 2021 05:35:03 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.fpga
Date: Tue, 14 Dec 2021 05:35:03 -0800 (PST)
In-Reply-To: <aPf*dzRyy@news.chiark.greenend.org.uk>
Injection-Info: google-groups.googlegroups.com; posting-host=2607:f2c0:e344:3f5:655a:ba11:a5d9:369a;
posting-account=pcMqhQoAAAATrGFhg20e9bXob643rZ-7
NNTP-Posting-Host: 2607:f2c0:e344:3f5:655a:ba11:a5d9:369a
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com>
<aPf*tX2xy@news.chiark.greenend.org.uk> <fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com>
<aPf*dzRyy@news.chiark.greenend.org.uk>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <2187e4d3-bb57-4885-9dfc-c68927f12a6fn@googlegroups.com>
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
From: chr...@chrisada.co.uk (Chris Adams)
Injection-Date: Tue, 14 Dec 2021 13:35:03 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 29
 by: Chris Adams - Tue, 14 Dec 2021 13:35 UTC

On Tuesday, 9 November 2021 at 15:24:51 UTC-5, Theo wrote:
> Chris Adams wrote:
>
> > Good idea. We tried using a design partition, but even the elaboration
> > stage uses over 120GB of memory before crashing. I may try setting up an
> > extremely large swap file.
> I wonder if the generate statement is causing the trouble, and whether just
> having a pile of flat instantiations would be any different?
>
> One other thing you could try, if the I/O isn't too troublesome, is a tree
> of instantiations. module A2 contains two instances of the module A, module
> A4 contains two instances of A2, module A8 two of A4, etc. That way it
> keeps the complexity at each level of hierarchy down. If the synthesiser is
> blowing up at the elaboration stage it could help if the elaboration in a
> specific module is within limits. I've not tried it though.
>
> I've had Stratix 10 builds need >16GB but <64GB, for what it's worth, but
> these weren't super full designs.
>
> Theo
This is also a good idea. We tried this, but it didn't help either.

We were eventually able to get the design to complete map, but it required a 500GB swap file, and took 2 days or so to build!!!

I don't think this is really a good solution, memory access on the system swap is extremely slow.

Anyway the design now fails at the route stage, saying the design cannot be routed... Device resource usage is around 60% following map.

Chris

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<04f71715-47a9-473b-8a20-eae8417df531n@googlegroups.com>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=257&group=comp.arch.fpga#257

  copy link   Newsgroups: comp.arch.fpga
X-Received: by 2002:ac8:7f4d:: with SMTP id g13mr7390036qtk.173.1643398596665;
Fri, 28 Jan 2022 11:36:36 -0800 (PST)
X-Received: by 2002:ac8:5c49:: with SMTP id j9mr1844871qtj.297.1643398596502;
Fri, 28 Jan 2022 11:36:36 -0800 (PST)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.arch.fpga
Date: Fri, 28 Jan 2022 11:36:36 -0800 (PST)
In-Reply-To: <2187e4d3-bb57-4885-9dfc-c68927f12a6fn@googlegroups.com>
Injection-Info: google-groups.googlegroups.com; posting-host=142.76.30.252; posting-account=pcMqhQoAAAATrGFhg20e9bXob643rZ-7
NNTP-Posting-Host: 142.76.30.252
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com>
<aPf*tX2xy@news.chiark.greenend.org.uk> <fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com>
<aPf*dzRyy@news.chiark.greenend.org.uk> <2187e4d3-bb57-4885-9dfc-c68927f12a6fn@googlegroups.com>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <04f71715-47a9-473b-8a20-eae8417df531n@googlegroups.com>
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
From: chr...@chrisada.co.uk (Chris Adams)
Injection-Date: Fri, 28 Jan 2022 19:36:36 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 31
 by: Chris Adams - Fri, 28 Jan 2022 19:36 UTC

On Tuesday, 14 December 2021 at 08:35:06 UTC-5, Chris Adams wrote:
> On Tuesday, 9 November 2021 at 15:24:51 UTC-5, Theo wrote:
> > Chris Adams wrote:
> >
> > > Good idea. We tried using a design partition, but even the elaboration
> > > stage uses over 120GB of memory before crashing. I may try setting up an
> > > extremely large swap file.
> > I wonder if the generate statement is causing the trouble, and whether just
> > having a pile of flat instantiations would be any different?
> >
> > One other thing you could try, if the I/O isn't too troublesome, is a tree
> > of instantiations. module A2 contains two instances of the module A, module
> > A4 contains two instances of A2, module A8 two of A4, etc. That way it
> > keeps the complexity at each level of hierarchy down. If the synthesiser is
> > blowing up at the elaboration stage it could help if the elaboration in a
> > specific module is within limits. I've not tried it though.
> >
> > I've had Stratix 10 builds need >16GB but <64GB, for what it's worth, but
> > these weren't super full designs.
> >
> > Theo
> This is also a good idea. We tried this, but it didn't help either.
>
> We were eventually able to get the design to complete map, but it required a 500GB swap file, and took 2 days or so to build!!!
>
> I don't think this is really a good solution, memory access on the system swap is extremely slow.
>
> Anyway the design now fails at the route stage, saying the design cannot be routed... Device resource usage is around 60% following map.
>
> Chris

Just a quick update - We were able to further reduce synthesis memory usage by lowering the number of cores used for compilation. We went down to 4 cores and memory usage was below 128GB with only a small impact on build performance.

Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

<zBe*iJrFy@news.chiark.greenend.org.uk>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=258&group=comp.arch.fpga#258

  copy link   Newsgroups: comp.arch.fpga
Path: i2pn2.org!rocksolid2!i2pn.org!news.niel.me!nntp.terraraq.uk!nntp-feed.chiark.greenend.org.uk!ewrotcd!.POSTED!not-for-mail
From: theom+n...@chiark.greenend.org.uk (Theo)
Newsgroups: comp.arch.fpga
Subject: Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design
Date: 28 Jan 2022 21:41:53 +0000 (GMT)
Organization: University of Cambridge, England
Lines: 14
Message-ID: <zBe*iJrFy@news.chiark.greenend.org.uk>
References: <395ee556-531f-4ec0-8757-caeb183e3009n@googlegroups.com> <aPf*tX2xy@news.chiark.greenend.org.uk> <fc835886-90ae-45c4-8fea-76196025419an@googlegroups.com> <aPf*dzRyy@news.chiark.greenend.org.uk> <2187e4d3-bb57-4885-9dfc-c68927f12a6fn@googlegroups.com> <04f71715-47a9-473b-8a20-eae8417df531n@googlegroups.com>
NNTP-Posting-Host: chiark.greenend.org.uk
X-Trace: chiark.greenend.org.uk 1643406115 21203 212.13.197.229 (28 Jan 2022 21:41:55 GMT)
X-Complaints-To: abuse@chiark.greenend.org.uk
NNTP-Posting-Date: Fri, 28 Jan 2022 21:41:55 +0000 (UTC)
User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (Linux/3.16.0-11-amd64 (x86_64))
Originator: theom@chiark.greenend.org.uk ([212.13.197.229])
 by: Theo - Fri, 28 Jan 2022 21:41 UTC

Chris Adams <chris@chrisada.co.uk> wrote:
> Just a quick update - We were able to further reduce synthesis memory
> usage by lowering the number of cores used for compilation. We went down
> to 4 cores and memory usage was below 128GB with only a small impact on
> build performance.

That's good to know, I hadn't thought of that. Makes sense - reduces
parallelism but reduces copies of the working set.

I get the impression Quartus spends a fair chunk of its time not being
parallel (Amdahl's law) - Quartus Pro is a bit better at being parallel, but
I think it still prefers fewer cores with a higher clock.

Theo


devel / comp.arch.fpga / Re: Quartus II Synthesis - System Memory Issues for Large Stratix 10 Design

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor