SHAREDMEMADDRESS BDE Configuration Parameter
Borland Database Engine (BDE) is a data access technology introduced by Borland a long time ago with Borland C++ and later on used by Delphi as a way of unifying access to DB data sources. At that time it had some interesting features that gave it some edge over ODBC for instance. For instance, if you followed some best practices, you were able to write applications accessing heterogeneous DB with the same client code (fat clients were common at the time). Also data access was fast, as Delphi was and still is (for Win32 applications). Over the time additional data access technologies surfaced but many legacy applications were kept accessing data over BDE native drivers. Incompatibilities can start to appear with new OS. Also, BDE is a Black Box. Documentation is scarce and it is no longer supported (so if you are using newer databases, chances are that you cannot access some of its new features or that somethings that worked before on previous DB engines do not work with the new versions. So when you or a customer decides to upgrade a DB, careful analysis and testing has to be performed.
After a long period of analysing stability problems of custom apps at a customer I have found the reason for some random errors "$210D: Shared Memory Conflict" occurring on Windows Server 2008 (that did not occur on another deployment server Windows 2003). The reason was traced to the address space layout randomization (ASLR) mechanism. For Windows, ASLR appeared first on Vista and aimed to introduce some randomness into the load address of some specially marked EXEs and DLLs (with the ASLR bit on) into the 32-bit address space of a Win32 process. For these operating systems some of OS DLLs are already marked with those flags. I am talking of DLLs like NTDLL.DLL, KERNEL32.DLL, etc.
To make a very long story short:
- BDE by default is configured to allocate the shared memory buffer it uses at 6BDE (0x6BDE0000, for Windows NT and above). This is inside the region used by the ASLR mechanism (above 0x50000000). This address is controlled by the SHAREDMEMADDRESS changeable in the BDE Administrator Control Panel applet or in the registry ([HKEY_LOCAL_MACHINE\SOFTWARE\Borland\Database Engine\Settings\SYSTEM\INIT] key).
- Every time a reboot is performed on the server a new randomization occurs and DLLs can from then on start to load on a different zone than the preferred load address. If a DLL is loaded into the 6BDE zone (or whatever address is set in SHAREDMEMADDRESS above 5000) then BDE tries to allocate the SHAREDMEMSIZE buffer into another region and typically succeeds. I haven't reached to the point of fully understanding the algorithm used for this rellocation since I was not given enough effort to debug this BDE behaviour (assembly).
- When in the same BDE client (i.e. same Windows Session on a PC with BDE installed locally) you open a second BDE app it will never be able to locate successfully the shared memory block in the rellocated address and will fail miserably with the "$210D: Shared memory conflict". Again I did not fully understand why this failure occurs (i.e. what kind of search is done and why it fails to locate a block of memory previously allocated by the same BDE code). A little more time spent debugging this should allow a full comprehension of this issue. But the customer needed a quick fix, so I had to move on.
The solution:
First list all BDE applications that a specific user will use. Using some memory mapping tool like VMMap, analyse all your BDE applications for a specific user. Find a region out of the ASLR region (below 0x50000000 minus the shared memory size configured in the BDE Admin) that is not being used consistently for every application. Notice that Delphi apps load at a preferred address (AFAIR 0x40000000) by default and this can be changed in Delphi compiler options. Also notice that old DLLs like BDE ones (IDAPI32.DLL) are not marked with the ASLR bit and as such they load at the preferred address stated in the file (PFE), unless another DLL is already loaded there.
After that analysis is done choose a valid address for SHAREDMEMADDRESS like 3BDE. But let me explain which addresses are valid addresses: Valid values for Windows NT and above are from 1000 to 7F00 but, as you have seen, if you are using Vista, Windows Server or any other OS that has the ASLR mechanism working, do NOT leave this setting at its default setting 6BDE. It could be a matter of time before you start having these errors reported from the users. The worst thing is that you will not be able to reproduce the issue easily on other test machines.
To locate the BDE shared memory location in VMMap, look for a block of memory with the indicated SHAREDMEMSIZE (if needed, close all BDE apps and change SHAREDMEMSIZE to a unique value so that you are able to spot it from other blocks).
I was able to reproduce this error in XP (SP2) by doing the reverse procedure and this was important to conclude that the reason for the error had been found. To reproduce the error you can follow this steps: Analyse the memory map for a BDE app, locate a DLL with the ASLR bit off (like IDAPI32.DLL) and take note of its load address. If it is in the valid range for SHAREDMEMADDRESS values (described above) configure this parameter to the same value. If not choose another DLL in the same conditions. Close all BDE applications. Open the first one and notice that the shared memory block is not being allocated at the configured address because it collides with that DLL (I have described how above). Open the same BDE application again and the $210D error will surface even in XP.
Note: ProcMon will allow you to detect if a process is ASLR activated or not (add the right column to the DLLs list for instance).
SHAREDMEMSIZE
Oh and BTW, if the error "$2501: Insufficient memory for this operation" is appearing to some of other users, increase the SHAREDMEMSIZE to 8192 or even a greater value (the maximum is 60000K; the minimum and default is 2048). Each BDE app consumes a fixed part of this shared memory buffer and a bigger buffer will allow for more applications to be open at the same time. This will increase each BDE EXE footprint in the OS after being loaded into memory but this should not be a problem if you have still enough virtual memory available in the OS. Another interesting thing i have discovered in the process was that by using the maximum value for SHAREDMEMSIZE I managed to open the maximum number of simultaneous BDE clients in a Windows session: 48. This was something I have also learned during this time. The limit to the number of BDE apps that can be opened at the same time is described in the "BDE Limits" help topic of BDEADMIN.HLP.
I leave you with a tip: Migrate this BDE apps to newer technologies ASAP.
You'll be doing yourself and your customer a big favor. Problems are around the corner and if you have to spot them by yourself you are alone in the dark (no support, no technical documentation, no source code, etc).
For details on these BDE parameters and another ones you can look at the BDEADMIN.HLP help file.
quarta-feira, 23 de dezembro de 2009
What I have learned in 6 weeks at Romania
For professional reasons I had to spend some time at Romania. Before the cache clears out I wrote some notes for posterity.
What I have learned in 6 weeks in Romania:
- Romanian is similar to Portuguese. Both are Latin languages (this I knew before). Even like that you have a hard time understanding what they are saying. Maybe with some more time and some investigation on sounds and common connecting words things are easier. The written form it is said to be easier to understand but I haven't tested this yet.
- There is no google news for Romania as we have for PT.
- The Romanian guys I knew are nice people but in their own words most Romanian are rude and poorly educated.
- Family violence seems to be even more common than in PT. And guess who wins the stats for aggression: Men.
- There are more women than men. Men have emigrated.
- Travelling in the subway is safe and nice to the sight :) women take care of themselves even for the most casual occasions.
- Food is nice, similar to Portuguese when compared. Not so healthy though.
- Fish is rare and expensive. They have a river fish and several strange species (to me).
- Plastic money are not credit cards. Paper money is made from plastic and has always a transparent part.
- Food is cheap (when compared to portugal - average meal for 7€).
- Beer is abailable at 0.5l portions or greater. No "minis" (20cl).
- S i l v a is a Romanian brand of beer.
- Caru cu bere is one of the finest places to dinner.
- Papanaçi is a nice dessert.
- Bucharest is huge, grey and its streets are not very well preserved.
- No highways out of Bucharest, as in Portugsl 20 years ago.
- Buildings from the communist period are not preserved at all and are ugly, grey and falling apart. In Portugal owners would be notified for making recuperation in a timely fashion and If they wouldn't abide they would lose ownership to the city hall or be demolished.
- Streets do have holes. Sometimes there are craters on them.
- The house of Ceausescu is the second largest building in the world after the Pentagon.
- There is an underground tunnel from one face to another.
- There are several underground levels some of them not yet finished.
- The parliament operates on part of it.
- Buildings around belonged to some of the secret police people and are now dedicated to national institutes.
- Most of the cars are old, very old. Lots of Dacia (similar to old Renaults).
- Bucharest is a busy city (lots of traffic).
- Air polution is a real problem - I was not used to it at the start.
- There are lots of Dacia cars which now still belongs to the Renault group.
- There is no public connection by subway from the airport to the city centre.
- The Otopeni airport departures zone is small: one corridor only.
- At summer it is hot and dry. At winter it is really cold! Portugal rarely goes below zero.
- As a consequence, houses are very well isolated. Finishing details are not very well crafted for what i've seen (but for what I've seen at Macau, not worse).
- It is said that there are lots of corruption.
- August is a bad time to visit because all the bad Romanians come back to their homes to visit families and the bad guys at prison (to be confirmed).
- If you are investing or buying everyone will try to sell you something (to get a comission?).
- Cars are cheap: They are not heavily taxed by the government like in PT where they put VAT over a government tax (also illegal in the UE as it is said).
- The main church is the Christian Orthodox church.
- It is common that when crossing a church door people make the cross sign on themselves for 3 times. Even younger people do this.
- An high percentage of the population is gipsy. Lots of them are still nomads (unlike in Portugal where some of them are
- Generally speaking: Romanian girls are skinny, at least to say. And they do take care of their appearence. I've seen some fat ones but obesity is definitely not a national problem there... yet (as it starts to be in Portugal). McDonalds and other plastic junk food starts to reign there.
- I found everyone still marked by the communist period. It was only in 1989 that freedom came and as such, younger people than me are marked by those events which for me is strange. I don't remember living in the dictatorship period in Portugal because... I was not in Portugal at that time.
- During the communist period everything was produced internally at Romania: cars, equipments, etc. After the end of this period some industries were abandoned.
- During the communist period electricity was sometimes cut at night.
- In Bucharest, enormous amounts of land were kept reserved to city parks with lakes, trees, bike trails, etc.
If I remember some more things I'll post them here.
If you feel anything is offensive or not true please let me know. I might be wrong or have had a wrong perception of the things there. Also, my English could be better.
What I have learned in 6 weeks in Romania:
- Romanian is similar to Portuguese. Both are Latin languages (this I knew before). Even like that you have a hard time understanding what they are saying. Maybe with some more time and some investigation on sounds and common connecting words things are easier. The written form it is said to be easier to understand but I haven't tested this yet.
- There is no google news for Romania as we have for PT.
- The Romanian guys I knew are nice people but in their own words most Romanian are rude and poorly educated.
- Family violence seems to be even more common than in PT. And guess who wins the stats for aggression: Men.
- There are more women than men. Men have emigrated.
- Travelling in the subway is safe and nice to the sight :) women take care of themselves even for the most casual occasions.
- Food is nice, similar to Portuguese when compared. Not so healthy though.
- Fish is rare and expensive. They have a river fish and several strange species (to me).
- Plastic money are not credit cards. Paper money is made from plastic and has always a transparent part.
- Food is cheap (when compared to portugal - average meal for 7€).
- Beer is abailable at 0.5l portions or greater. No "minis" (20cl).
- S i l v a is a Romanian brand of beer.
- Caru cu bere is one of the finest places to dinner.
- Papanaçi is a nice dessert.
- Bucharest is huge, grey and its streets are not very well preserved.
- No highways out of Bucharest, as in Portugsl 20 years ago.
- Buildings from the communist period are not preserved at all and are ugly, grey and falling apart. In Portugal owners would be notified for making recuperation in a timely fashion and If they wouldn't abide they would lose ownership to the city hall or be demolished.
- Streets do have holes. Sometimes there are craters on them.
- The house of Ceausescu is the second largest building in the world after the Pentagon.
- There is an underground tunnel from one face to another.
- There are several underground levels some of them not yet finished.
- The parliament operates on part of it.
- Buildings around belonged to some of the secret police people and are now dedicated to national institutes.
- Most of the cars are old, very old. Lots of Dacia (similar to old Renaults).
- Bucharest is a busy city (lots of traffic).
- Air polution is a real problem - I was not used to it at the start.
- There are lots of Dacia cars which now still belongs to the Renault group.
- There is no public connection by subway from the airport to the city centre.
- The Otopeni airport departures zone is small: one corridor only.
- At summer it is hot and dry. At winter it is really cold! Portugal rarely goes below zero.
- As a consequence, houses are very well isolated. Finishing details are not very well crafted for what i've seen (but for what I've seen at Macau, not worse).
- It is said that there are lots of corruption.
- August is a bad time to visit because all the bad Romanians come back to their homes to visit families and the bad guys at prison (to be confirmed).
- If you are investing or buying everyone will try to sell you something (to get a comission?).
- Cars are cheap: They are not heavily taxed by the government like in PT where they put VAT over a government tax (also illegal in the UE as it is said).
- The main church is the Christian Orthodox church.
- It is common that when crossing a church door people make the cross sign on themselves for 3 times. Even younger people do this.
- An high percentage of the population is gipsy. Lots of them are still nomads (unlike in Portugal where some of them are
- Generally speaking: Romanian girls are skinny, at least to say. And they do take care of their appearence. I've seen some fat ones but obesity is definitely not a national problem there... yet (as it starts to be in Portugal). McDonalds and other plastic junk food starts to reign there.
- I found everyone still marked by the communist period. It was only in 1989 that freedom came and as such, younger people than me are marked by those events which for me is strange. I don't remember living in the dictatorship period in Portugal because... I was not in Portugal at that time.
- During the communist period everything was produced internally at Romania: cars, equipments, etc. After the end of this period some industries were abandoned.
- During the communist period electricity was sometimes cut at night.
- In Bucharest, enormous amounts of land were kept reserved to city parks with lakes, trees, bike trails, etc.
If I remember some more things I'll post them here.
If you feel anything is offensive or not true please let me know. I might be wrong or have had a wrong perception of the things there. Also, my English could be better.
terça-feira, 22 de dezembro de 2009
Delphi to .Net converters
Some tools I am keeping in my watch list to convert Delphi code to .Net:
Source: http://www.delphitodotnet.com/Articles/IsthereanautomatedwaytoconvertDelphitoCSharp.aspx
- Delphi2CS: http://www.netcoole.com/delphi2cs.htm
- TurnSharp: https://www.turnsharp.com/home.aspx
- Metex (consulting): http://www.metex.com/?gclid=CMnAueKs654CFaBb4wodsznuJg
Source: http://www.delphitodotnet.com/Articles/IsthereanautomatedwaytoconvertDelphitoCSharp.aspx
MS Tag tags
MS Tag seems to be an interesting idea. I have known some similar technologies (3D bar codes) so I still don't get the real advantages of this one since 3D tags can be produced and read for free a long time ago. Maybe it is the integration with each mobile phone browser that MS will be providing for free.
MS tag allows people linking easily to web contents only by scanning (on phones) the image displayed on catalogs, wall advertisements, etc. This will avoid you having to take note of a link or having to type it later on in your browser. Typing by hand a link is cumbersome. Even short ones.
You login into the site (MSN credentials will work) and you can start creating your tags by providing a sentence and an URL to what you want to link to. After that the tag is listed on a grid. On the render grid column you can create several versions of the tag to embed in your site, publishing magazines, etc.
Tip: The first time you try to render, a different popup appears that forces you to read the publishing guidelines. After doing this for the first time, the next time you press render a popup menu with options to render the currently selected tag line apears: PDF, png, tiff, jpg, etc. are some options available.
MS tagging publisher guidelines can be found at: http://www.microsoft.com/tag/content/why/guidelines.aspx
The bad news is that you'll have to pay for each generated coloured tag when the service goes off Beta.
Also, I've tried the above logo with the iPhone. Open the MS Tag app, photographed the logo on my laptop LCD, chose "Use" action and there you go: A browser page with this blog on the iPhone browser appeared. Nice and sweet and with no sweat!
Azure@Curia
Recently I was present at the Azure@Curia at the "WRC Curia".
Details of the event can be found at: http://events.linkedin.com/Azure-Curia/pub/179391
This was a special event for me mainly for personal reasons. My grandmother's house is 50 meters away from the place the event was and I was able to spot it through the Window of the facilities. The place were WRC was built was a long time ago a rich man's house and my grandmother's house was one of the two or three separated houses of the caretakers of a farm. By the way, a now closed guesthouse at Curia belonged to my other grandfathers, who lived nearby in their final years.
Regarding the event, I found it really interesting. The cloud is a powerful thing, enabling everyone, even small companies, to create products and sell them without having to bother with complex infra-structure partnerships, and so on. Deployment is immediate. Regarding the cloud I have some security concerns (data privacy) and backup concerns (remember the Sidekick recent loss of data) but things will be clearer as times goes by: privacy statements from M$, backup policies, etc.
Azure seems an interesting thing mainly for anyone who already masters the .Net platform. If you have follow some guidelines while developing, applications can be ported to the Cloud almost effortlessly. Applications can be developed to work online and offline if needed.
Interesting things I have never thought of were:
- Quality is even more important than ever. Now that you pay for bytes transferred to the DB and back, you don't want to have a development team that does not develop unit tests thoroughly and that deploys without running the suite of system tests against the product locally. More than ever, you have to fire that sloppy programmer of yours that made that infinite loop while reading from the Azure DB.
- There is a market for development of auxiliary tools that complement what MS is offering for managing the applications in the cloud. For instance, there is no way or configuration possible to be made to warn you when a certain amount of data was trasnferred to the cloud (eventually to warn you of a bug in the software you have deployed to prevent heavy charges in the end of the month).
- You can have a single application developed to the cloud, and have several customers using it in the very same database. Customization can be made by skinning the application along with the authenticated user. Interesting stuff no? So every time you fix a bug or improve your product and deploy it to the Cloud, you are in fact updating every customer of yours. No need to remote desktop everyone of them to install on their servers or to do the update at several ISPs to cover all of your customers sites. Again, extra care has to be put in the final stages, testing before deploying. But this is not something new. But I liked the mechanism of quickly being able to reverse a deployment in the cloud, just in case.
More about this will be coming, as soon as I remember it.
The agenda of the event was:
14h30 ‐ Cloud Computing: A plataforma Azure
Luís Alves Martins, Architect Advisor ‐ Microsoft
15h45 ‐ Armazenamento na Cloud
José António Silva – R&D Director ‐ Devscope
17h00 ‐ Windows Azure na prática
Luís Alves Martins/Carlos Fernandes
Details of the event can be found at: http://events.linkedin.com/Azure-Curia/pub/179391
This was a special event for me mainly for personal reasons. My grandmother's house is 50 meters away from the place the event was and I was able to spot it through the Window of the facilities. The place were WRC was built was a long time ago a rich man's house and my grandmother's house was one of the two or three separated houses of the caretakers of a farm. By the way, a now closed guesthouse at Curia belonged to my other grandfathers, who lived nearby in their final years.
Regarding the event, I found it really interesting. The cloud is a powerful thing, enabling everyone, even small companies, to create products and sell them without having to bother with complex infra-structure partnerships, and so on. Deployment is immediate. Regarding the cloud I have some security concerns (data privacy) and backup concerns (remember the Sidekick recent loss of data) but things will be clearer as times goes by: privacy statements from M$, backup policies, etc.
Azure seems an interesting thing mainly for anyone who already masters the .Net platform. If you have follow some guidelines while developing, applications can be ported to the Cloud almost effortlessly. Applications can be developed to work online and offline if needed.
Interesting things I have never thought of were:
- Quality is even more important than ever. Now that you pay for bytes transferred to the DB and back, you don't want to have a development team that does not develop unit tests thoroughly and that deploys without running the suite of system tests against the product locally. More than ever, you have to fire that sloppy programmer of yours that made that infinite loop while reading from the Azure DB.
- There is a market for development of auxiliary tools that complement what MS is offering for managing the applications in the cloud. For instance, there is no way or configuration possible to be made to warn you when a certain amount of data was trasnferred to the cloud (eventually to warn you of a bug in the software you have deployed to prevent heavy charges in the end of the month).
- You can have a single application developed to the cloud, and have several customers using it in the very same database. Customization can be made by skinning the application along with the authenticated user. Interesting stuff no? So every time you fix a bug or improve your product and deploy it to the Cloud, you are in fact updating every customer of yours. No need to remote desktop everyone of them to install on their servers or to do the update at several ISPs to cover all of your customers sites. Again, extra care has to be put in the final stages, testing before deploying. But this is not something new. But I liked the mechanism of quickly being able to reverse a deployment in the cloud, just in case.
More about this will be coming, as soon as I remember it.
The agenda of the event was:
14h30 ‐ Cloud Computing: A plataforma Azure
Luís Alves Martins, Architect Advisor ‐ Microsoft
15h45 ‐ Armazenamento na Cloud
José António Silva – R&D Director ‐ Devscope
17h00 ‐ Windows Azure na prática
Luís Alves Martins/Carlos Fernandes
Subscrever:
Mensagens (Atom)
