Обсуждение: build environment: a different makefile
Currently trying to "enhance" the way we can make binaries that run on Solaris. One thing I found was a scalability bottleneck in the use of the ProcArrayLock. (this one has also been reported by a couple of my colleagues). One big user of this lock is GetSnapshotData. After it has taken this lock it does its work and releases it again. While it is holding the lock it is not doing any system calls and the lock holding process is barely preempted. The only way to make this code faster is making the code use less CPU cycles to achieve its goal. One way is having the compiler do some strong code in-lining. The SunStudio compiler we are using fortunately has an option for this. Unfortunately there are restrictions. One restriction I face is its inability to deal with "ld -r"s. These are used in the build environment to create all the SUBSYS.o object files. I was hoping someone in the community already has a makefile that "just" creates object files from C-sources directly that I can use to try out the effect of in-lining to the performance of postgres. Any other hints to achieve my goal are welcome too, of-course. Please note that in-lining is done in both the compiler and the linker. Thanks, Paul ------------------------------------------------------------------------ --------------------- Paul van den Bogaard Paul.vandenBogaard@sun.com ISV-E -- ISV Engineering, Opensource Engineering group Sun Microsystems, Inc phone: +31 334 515 918 Saturnus 1 extentsion: x (70)15918 3824 ME Amersfoort mobile: +31 651 913 354 The Netherlands fax: +31 334 515 001
Paul van den Bogaard wrote: > The SunStudio compiler we are using fortunately has an option for > this. Unfortunately there are restrictions. One restriction I face is > its inability to deal with "ld -r"s. These are used in the build > environment to create all the SUBSYS.o object files. > > I was hoping someone in the community already has a makefile that > "just" creates object files from C-sources directly that I can use to > try out the effect of in-lining to the performance of postgres. I don't know if anyone has a makefile for it, but the following seems to work for me: pgsql/src/backend$ cc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels-fno-strict-aliasing -g -L../../src/port -Wl,-rpath,'/home/peter/devel/pg83/pg-install/lib' -Wl,-E $(find-name "*.o" | grep -v SUBSYS | grep -v conversion_procs) ../../src/timezone/SUBSYS.o ../../src/port/libpgport_srv.a-lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lcrypt -ldl -lm -lldap -o postgres If you find that the optimizations you are hoping for are useful, I'm sure we could put an option of that sort somewhere in the makefiles. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut <peter_e@gmx.net> writes:
> I don't know if anyone has a makefile for it, but the following seems to work
> for me:
> pgsql/src/backend$ cc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement
-Wendif-labels-fno-strict-aliasing -g -L../../src/port -Wl,-rpath,'/home/peter/devel/pg83/pg-install/lib' -Wl,-E
$(find-name "*.o" | grep -v SUBSYS | grep -v conversion_procs) ../../src/timezone/SUBSYS.o
../../src/port/libpgport_srv.a-lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lcrypt -ldl -lm -lldap -o postgres
> If you find that the optimizations you are hoping for are useful, I'm sure
> we could put an option of that sort somewhere in the makefiles.
I've sometimes wondered whether the SUBSYS.o files really offer any
advantage compared to just linking all the individual .o files. They
certainly eat disk space, but perhaps they save some time ... or perhaps
not, especially in a one-off build.
I suppose that we might fall foul of command line length limits on
some platforms :-(. The output of your find command amounts to nearly
11000 characters in HEAD.
regards, tom lane
On Feb 6, 2008 11:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > I've sometimes wondered whether the SUBSYS.o files really offer any > advantage compared to just linking all the individual .o files. They > certainly eat disk space, but perhaps they save some time ... or perhaps > not, especially in a one-off build. Getting rid of them would certainly make building OSX universal binaries easier. > I suppose that we might fall foul of command line length limits on > some platforms :-(. The output of your find command amounts to nearly > 11000 characters in HEAD. What do other large build systems do? /D
On Thu, Feb 07, 2008 at 08:09:24AM +0000, Dave Page wrote: > On Feb 6, 2008 11:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > > I've sometimes wondered whether the SUBSYS.o files really offer any > > advantage compared to just linking all the individual .o files. They > > certainly eat disk space, but perhaps they save some time ... or perhaps > > not, especially in a one-off build. > > Getting rid of them would certainly make building OSX universal binaries easier. > > > I suppose that we might fall foul of command line length limits on > > some platforms :-(. The output of your find command amounts to nearly > > 11000 characters in HEAD. > > What do other large build systems do? FWIW, the MSVC build ends up writing the list of object files to a temp file and then having the linker read that list. (This is all done behind the scenes though, it's not something we made up) IIRC the gcc linker can also take the commandline from a file instead of the actual commandline, which should be workable I think. //Magnus
Am Donnerstag, 7. Februar 2008 schrieb Dave Page: > What do other large build systems do? I strongly suspect that they just fail unless run with GNU tools or some other well-defined tool set. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter Eisentraut napsal(a): > Paul van den Bogaard wrote: >> The SunStudio compiler we are using fortunately has an option for >> this. Unfortunately there are restrictions. One restriction I face is >> its inability to deal with "ld -r"s. These are used in the build >> environment to create all the SUBSYS.o object files. >> >> I was hoping someone in the community already has a makefile that >> "just" creates object files from C-sources directly that I can use to >> try out the effect of in-lining to the performance of postgres. > > I don't know if anyone has a makefile for it, but the following seems to work > for me: > > pgsql/src/backend$ cc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels-fno-strict-aliasing -g -L../../src/port -Wl,-rpath,'/home/peter/devel/pg83/pg-install/lib' -Wl,-E $(find-name "*.o" | grep -v SUBSYS | grep -v conversion_procs) ../../src/timezone/SUBSYS.o ../../src/port/libpgport_srv.a-lxslt -lxml2 -lpam -lssl -lcrypto -lgssapi_krb5 -lcrypt -ldl -lm -lldap -o postgres > > If you find that the optimizations you are hoping for are useful, I'm sure > we could put an option of that sort somewhere in the makefiles. Peter, Suns studio performs inline optimization on -xO3 level. Optimization levels are different from GCC. Maximal level is -xO5. I think Paul plays with xipo flag which requires at least -xO4. See http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view Zdenek
Zdenek is right. Normal inlining using -O3 or higher means inlining within the source file. I currently try to see the effect of inlining over all the sources. The -xipo flag is specific to the Sun Studio (version 12) suite. It creates large objects that now include meta data for the final linking stage. During this linking stage all this stuff is read, analyzed an the object files are changed. Later this can be expanded by compiler profiling and/or other trickery. BTW the actual binary does *not* include all that metadata and is just bigger due to all that code being inlined in those objects. This trick has been done for other applications and we find in general significant approvement. Not a guarantee though ... However my "suite" has a dependecy on dtrace so the initial idea I saw from Peter is not complete. Currently creating a quick and dirty build script. Just to get that "ipo"-ing done and tested. I'll keep you all updated on the results. If we decide it is too good to be excluded can we start thinking about an adaption of the build environement. Hope this sound fair and acceptable to all. Thanks so far for all that feedback Cheers Paul BTW I build and test on a 16 SPARC @1350MHz V890. 64 bit mode, currently 16GB of shared memory, and 7 disk arrays (RAID0). This to exclude the IO part of the equation as much as possible, so we can focus on CPU related matter: the need for faster (smarter) code pathes and/or scalability issues like ProcArrayLock contention. On 7-feb-2008, at 11:08, Zdenek Kotala wrote: > Peter Eisentraut napsal(a): >> Paul van den Bogaard wrote: >>> The SunStudio compiler we are using fortunately has an option >>> for this. Unfortunately there are restrictions. One restriction >>> I face is its inability to deal with "ld -r"s. These are used in >>> the build environment to create all the SUBSYS.o object files. >>> >>> I was hoping someone in the community already has a makefile >>> that "just" creates object files from C-sources directly that I >>> can use to try out the effect of in-lining to the performance of >>> postgres. >> I don't know if anyone has a makefile for it, but the following >> seems to work >> for me: >> pgsql/src/backend$ cc -O2 -Wall -Wmissing-prototypes -Wpointer- >> arith -Winline -Wdeclaration-after-statement -Wendif-labels -fno- >> strict-aliasing -g -L../../src/port -Wl,-rpath,'/home/peter/devel/ >> pg83/pg-install/lib' -Wl,-E $(find -name "*.o" | grep -v SUBSYS | >> grep -v conversion_procs) ../../src/timezone/SUBSYS.o ../../src/ >> port/libpgport_srv.a -lxslt -lxml2 -lpam -lssl -lcrypto - >> lgssapi_krb5 -lcrypt -ldl -lm -lldap -o postgres >> If you find that the optimizations you are hoping for are useful, >> I'm sure >> we could put an option of that sort somewhere in the makefiles. > > Peter, > Suns studio performs inline optimization on -xO3 level. > Optimization levels are different from GCC. Maximal level is -xO5. > I think Paul plays with xipo flag which requires at least -xO4. > > See > http://docs.sun.com/app/docs/doc/819-5265/bjapp?a=view > > Zdenek > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster ------------------------------------------------------------------------ --------------------- Paul van den Bogaard Paul.vandenBogaard@sun.com ISV-E -- ISV Engineering, Opensource Engineering group Sun Microsystems, Inc phone: +31 334 515 918 Saturnus 1 extentsion: x (70)15918 3824 ME Amersfoort mobile: +31 651 913 354 The Netherlands fax: +31 334 515 001
Magnus Hagander <magnus@hagander.net> writes:
> On Thu, Feb 07, 2008 at 08:09:24AM +0000, Dave Page wrote:
>> What do other large build systems do?
> FWIW, the MSVC build ends up writing the list of object files to a temp
> file and then having the linker read that list. (This is all done behind
> the scenes though, it's not something we made up) IIRC the gcc linker can
> also take the commandline from a file instead of the actual commandline,
> which should be workable I think.
Hmm. I'm not sure that's universal, but if it is then a simple
incremental improvement on what we're doing now would be to replace the
SUBSYS.o files with "subsys include files" that just list all the .o
files to be included.
regards, tom lane
Am Mittwoch, 6. Februar 2008 schrieb Paul van den Bogaard: > I was hoping someone in the community already has a makefile that > "just" creates object files from C-sources directly that I can use to > try out the effect of in-lining to the performance of postgres. This is now the default in 8.4devel. Let us know what you find out. -- Peter Eisentraut http://developer.postgresql.org/~petere/
Peter, finally I had a chance to check it out. One word: perfect! Thanks Paul On 25-feb-2008, at 19:09, Peter Eisentraut wrote: > Am Mittwoch, 6. Februar 2008 schrieb Paul van den Bogaard: >> I was hoping someone in the community already has a makefile that >> "just" creates object files from C-sources directly that I can use to >> try out the effect of in-lining to the performance of postgres. > > This is now the default in 8.4devel. Let us know what you find out. > > -- > Peter Eisentraut > http://developer.postgresql.org/~petere/ > > ---------------------------(end of > broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings ------------------------------------------------------------------------ --------------------- Paul van den Bogaard Paul.vandenBogaard@sun.com ISV-E -- ISV Engineering, Opensource Engineering group Sun Microsystems, Inc phone: +31 334 515 918 Saturnus 1 extentsion: x (70)15918 3824 ME Amersfoort mobile: +31 651 913 354 The Netherlands fax: +31 334 515 001