No longer able to compile from 5.026 to 5.069 [NO SOLUTION]

[benoitstjean]

Old compiler: 5.026
New compiler: 5.069
Device: PIC24HJ256GP206 / PIC24EP512GP806

Alright, so *another* problem going from 5.026 to 5.069:

In my code, I have a lot of fprintf() statements that prints stuff to a serial console on my PC. This takes-up a lot of space. However, each fprintf() statement is wrapped with an <#ifdef ENABLE_FPRINTF / #endif> switch so that I can disable all fprintf() messages therefore free-up some RAM, even ROM.

Also, using a second #ifdef "switch", I can make the code compile for either the PIC24HJ256GP206 or the PIC24EP512GP806:


Compile for PIC24HJ256GP206:

Using 5.026:

When the #ifdef statement is active (e.g. fprintf() statements are ignored): RAM is 76%, ROM is 77%
When the #ifdef statement is inactive (e.g. fprintf() statements are compiled): RAM is 77%, ROM is 88%


Using 5.069:

#ifdef statement is active (e.g. fprintf() statements are ignored): Error NOT ENOUGH RAM FOR ALL VARIABLES


Compile for PIC24EP512GP806:

Using 5.026:

When the #ifdef statement is active (e.g. fprintf() statements are ignored): RAM is 37%, ROM is 43%
When the #ifdef statement is inactive (e.g. fprintf() statements are compiled): RAM is 37%, ROM is 51%

Using 5.069:

When the #ifdef statement is active (e.g. fprintf() statements are ignored): RAM is 50%, ROM is 45%
When the #ifdef statement is inactive (e.g. fprintf() statements are compiled): RAM is 49%, ROM is 54%



So my observation here is that 5.069 drastically increases RAM usage for the exact same code, regardless of the PIC used.... As seen above, for the PIC24HJ, I get errors going from 5.026 to 5.069 and for the PIC24EP, it goes from 37% with 5.026 to 50% with 5.069.

Anyone has a valid / logic explanation?

Thanks,

Benoit

[EDIT] I know that I should have smaller functions and that's what I'll start to look at but still, it doesn't explain the 13% RAM difference from 5.026 to 5.069 with the same code.

[Ttelmah]

By default the compiler now compiles for speed, rather than ROM size. There is an option now to select for compact code instead.
#opt compress

[benoitstjean]

Hi TTelmah,

I looked in the CCS manual and there's not much information as to what >#opt compress> does.

So when you say that it 'compacts' the code, do you know a bit what it does or how it does it in order to not break the code?

I'm just asking out of curiosity.

[EDIT] Forgot.... where in the code should I put this? I know it should be at the begining but is there a location better than another?

Thanks again,

Benoit

[benoitstjean]

So I added '#opt compress' immediately after '#include <24HJ256GP206.h>' and I still cannot compile with 5.069 and still get "Not enough RAM for all variables" but 5.026 compiles just fine.

[EDIT] With 5.026, I don't use '#opt compress' and I compile and the code is 76% RAM and 77% ROM. Then going to 5.069 gives the error, even with '#opt compress'. So unless that command has to be put at a very specific place?

Benoit

[jeremiah]

If it were me, I would start by looking at the .SYM files for all the ones that compiled. Particularly to compare the PIC24EP512GP806 5.026 vs 5.069. That might give clues as to where the extra RAM is coming from.

My own experience has not seen such large RAM changes between versions in those ranges, but I also don't usually need things like PASS_STRINGS=IN_RAM and such.

If you can identify that, then it might help with finding a solution or at least finding a potential bug/design issue in the compiler.

You might consider posting the .SYM files here in case someone else can spot something you miss.

[benoitstjean]

Hi Jeremiah,

Thanks for the info. Oddly enough, CCS support also requested to look at the .SYM file but quite frankly, I have no idea what I am supposed to look for.

From the compiler, if I click VIEW->SYMBOLS, it does open-up a .SYM file which is what I presume you're requesting..... but the information doesn't seem to indicate anything relevant to me.... and I've never had to go through this excercise before so I need guidance.

Here is an excerpt of what I am seeing in the file:

W0 @SCRATCH
W0L _RETURN_
W0 -W2 @DIVS4848.P1
W0 -W1 @READ_ROM_MEMORY.P2
W0 @delay_ms1.P3
W0 -W1 @DIV3232B.P4

... and so on.

Then about half-way through the file, I see this (excerpt):

ROM Allocation:
000200 MUXCRCTable.call
00021C MUXCRCTable.data
0002C8 @const3207.call
0002DE @const3208.call
0002FC @const3209.call

... and so on.


So, is this the right file and if so, what do I need to look for?

Thanks again,

Benoit

[benoitstjean]

Ok. So I am looking at both SYM files. CCS support said to look past the 'W' registers section then I should see addresses and address ranges which should technically indicate the size, in bytes, used by whatever it is next to....

Given that both SYM files got compiled by different compilers, trying to do a file compare is quite complex because the data is not organized the same. I mean, the program I am using <DiffMerge> shows red makrs on the left side of the app to show where the differences are. The problem is that the entire column is red, litterally. The lines 'numbering' might be the same for a given line but the values are different.

Like for example, if I take line 999, this is what I see:

5.029 .SYM file:
999 591B move_window.n

5.069 .SYM file:
999 686B move_window.n

So the same line number, both use only one byte but the address has changed since it is from a different compiler. The entire file is seen as different.

Also, if I look in the 5.026 SYM file, the last lines before <ROM Allocation:> are:

5952-5953 SD_write_data.WriteCount
595A SD_cmd.cmd
595B SD_cmd.i
595C-595F SD_cmd.address
5960 SD_cmd.response
5962 wait_ready.response
5962-5963 SD_cmd.value
5964-5965 wait_ready.Timer
5966 SD_spi_write.data
5967 SD_spi_write.dummy
5974 SD_spi_read.command
5975 SD_spi_read.dummy
CC00-CFFF Stack

But if I take the same last lines from 5.069, this is what I get:

687C-687D SD_write_data.WriteCount
687D SD_cmd.i
687E-687F SD_receive_data.i
687E-6881 SD_cmd.address
6880 SD_receive_data.response
6882 SD_cmd.response
6884-6885 SD_cmd.value
6A7E wait_ready.response
6A80-6A81 wait_ready.Timer
6A80 SD_spi_write.data
6A81 SD_spi_write.dummy
6A8A SD_spi_read.command
6A8B SD_spi_read.dummy
7C00-7FFF Stack

So how / why is the stack address range that different?

Is there an easier way to do all of this? There's slightly more than 800 lines to compare and all have different 'address' values.

Thanks again,

Benoit

[jeremiah]

It looks like the sym file for 5.026 thinks the chip has 52k RAM and the 5.069 compiler thinks it has 32k RAM.

I can't view the datasheet, but how much RAM is that chip supposed to have?

As far as what to look for:

After all the W variables, the first normal address is 800. I would try to get both sym files up in some sort of side by side comparison, and scan through them looking for where large changes in address occur. both will start at 800, but at some point they will start to diverge. Then you can hone in on those spots and identify which variables are causing the large jumps in address.

I don't know of a super easy way to do it. Normally, I just buckle down and scan till I find a likely spot, then go line by line in that area. I would then make note of any different variables. From there I would look in the code and see how they are used. Floating point math and string libs are places where I might expect culprits.

The format of those symbol names is function.variable_name so they should be easy to cross reference in your code. The only tricky ones are the ones that CCS creates under the hood. Those can have odd names (hence the floating point and string libs comment).

[benoitstjean]

From Microchip's website: RAM Bytes 53,248.

I found a place where the RAM allocation jumps by like 1026 bytes all of a sudden in one file and only a few hundred in another place and CCS says that this is odd. The address prior to these two jumps in both files is off by 2 bytes only.

But _then_ it jumps from address x to x+1026 in one file and from x to x+400 in the other.

Also, in one file, let's say that x and x+1026 are two functions, in the other file, there's like 5 other lines in-between. That's very confusing.

CCS responded that they made memory allocation changes in 5.070 and 5.071 and some info I sent them (not in this post) are a concern to them. They are looking into it at the moment.

But going through the SYM files is quite the feat. Given that it is two different compiler versions, different mechanics is happening in the background therefore both SYM files are completely different.

As indicated in my previous pose, I am using DIFFMERGE to view the files side by side. There's a thin vertical bar on the left in the application which is a 'zoomed-out' view of the entire file. Then there's an outlined area which is a 'window' of what I'm seeing in the readable area of the screen (the two files).

Then, in that same bar, a red line appears at the location where a difference is detected. Usually, if I have two identical files with only one change, I would see the equivalent of one thin red line on the side making it easy to go to that change. In my case, because the SYM files are so long, the red marking is the equivalent of one pixel high and given that both files have differences in almost every line, it is not very helpful because that vertical bar on the left is red from top to bottom with only a few very thin white gaps where no changes were detected.

Even if I analyze line by line, the addresse (or address ranges) used-up by variables and functions are totally different from one file to the other plus the order of the lines in one file might be like A, B, C, D, E, F, G and in the other file, that order would look like A, D, G, E, B, C, M, K. A line in one file can be 10 lines down or 10 lines up in the other file.

It's like if I have a puzzle with 1000 pieces: both puzzles have the same image but the shapes of the parts are different and one has 2000 pieces. That's sort of what I'm facing.

Anyhow, I'll see what CCS comes-up with.

Benoit

[jeremiah]

Using a diff tool probably will only make it harder. The addresses are gonna be different. That's gonna highlight a ton of lines (as you mentioned). What should be reasonably the same is the actual symbols they use ( function_name.variable_name ) and you can usually start off by lining those up at address 800. Even in the snippets you posted earlier, you can see the similarities between the two files in terms of symbols. The order is *mostly* the same.

That said, if CCS is looking into it, they will probably get a fixed version to you. You might also make sure they know that the 5.069 version has the wrong RAM number for that chip. That's your primary problem and why you run out of RAM (aside from the allocation oddities). You might be able to fix it in the device editor if you have the IDE version of the compiler.

As a side note, I would definitely recommend that you at some point take a smaller project that you know the code really well on and look at how the symbol table allocates variables and functions for it. They SYM file is a great tool for finding programming bugs like accessing an array out of bounds. I've used it several times in the past. It does get easier to read with time, mostly an experience thing.

[benoitstjean]

Ok, so little update on this issue....

I went up to 5.071 and still have issues. I have 5.071, 5.069, 5.056 and 5.026 and only 5.026 works. When I say "works", I mean that I compile and burn the code and the circuit works as expected like it has always worked.

When I compile with any other version, the circuit works up to a certain point then stops receiving UART messages. Same exact code in both, just a different compiler.

I have also noticed huge discrepancies in RAM usage between versions for the exact same code going from 5.026 to 5.071:

5.026: 37%
5.056: 60%
5.069: 49%
5.071: 44%

So I am now back to 5.026 compiler inside the 5.071 IDE. I will make a new post for this...

Thanks all for your help,

Benoit

[jeremiah]

Yeah, 5.071 has been out for a bit, so it probably doesn't have any fixes in this regard. Did CCS support indicate they could fix the wrong RAM numbers? Did you try using the device editor to fix them yourself while you wait?

[benoitstjean]

Hi Jeremiah,

In order to find the RAM issues, they ask that I send them my files but I am not authorized by my employer unfortunately.

They have also requested that I 'compare' SYM files and LST files but the .LST files have around 130K lines (line numbers on the left).

But anyhow, I am now using the 5.071 IDE which enables me to use the __BUILDNUMBER__ option with the 5.026 compiler.

Not sure what is the latest CCS version, maybe I'll ask them to try a later version. Afterall, I _am_ paying for the license so I don't see why I'd pay for a buggy compiler when older versions work.

If I ever find a solution, I will post it here.

Thanks again,

Ben