Dynamic Memory Management
TOPICS COVERED:::
Dynamic Memory Management:
1. Understanding pointers,
2. usage of pointers,
3. arithmetic on pointers,
4. memory allocation,
5. memory management functions and operators,
6. debugging pointers - danglling pointers,
7. memory leaks
.POINTERS
To make full use of the C Programming language, you have to have a very good understanding of pointers. For most people it will take some time to fully understand pointers. So be patient. You have to learn pointers because they are used everywhere in the C language. Once you master the use of pointers, you will use them everywhere. OK, enough pep talk, let’s start.
Pointers are used (in the C language) in three different ways:
- To create dynamic data structures.
- To pass and handle variable parameters passed to functions.
- To access information stored in arrays. (Especially if you work with links).
Pointers are also used by experienced programmers to make the code more efficient and thus faster.
So why use pointers? Why don’t we use arrays to create data structures?
The answer is simple. With an array you have to declare its maximum size (for every dimension) at the beginning. Let’s say you create an array that can hold a maximum of twenty megabytes. When the array is declared, the twenty megabytes is claimed. Now this time you have only data for ten megabytes.
(A next time it could be fifteen megabytes or five megabytes). So in this case ten megabytes of memory is wasted, because only ten megabytes from the twenty is used.
This is where pointers come in. With pointers, you can create dynamic data structures. Instead of claiming the memory up-front, the memory is allocated (from the heap) while the program is running. So the exact amount of memory is claimed and there is no waste. Even better, memory not used will be returned to the heap. (Freed memory can be used for other programs).
Pointer basics
To better understand pointers, it sometimes helps to compare a “normal variable” with a pointer.
When a “normal variable” is declared, memory is claimed for that variable. Let’s say you declare an integer variable MYVAR. Four bytes of memory is set aside for that variable. The location in memory is known by the name MYVAR. At the machine level that location has a memory address.
A pointer differs in the way that a pointer is a variable that points to another variable. A pointer holds the memory address of that variable. That variable contains a value. Pointers are also called address variables because they contain the addresses of other variables.
Example: We have a piece of memory with a start address of 0×2000. That piece of memory contains a value and is named MYEXAMPLE.
(This is in fact a variable). We also have a pointer MYPOINT. In this case, our pointer MYPOINT contains the address 0×2000. This is the address of MYEXAMPLE so we say MYPOINT points to MYEXAMPLE. So there is the pointer and the value pointed to. (You can use a pointer in a certain way to get the value at the address to which the pointer points). Many novice programmers get pointers and their contents confused.
So from now on every pointer starts with the extension ptr_ . (for example: ptr_MYPOINT).
So from now on every pointer starts with the extension ptr_ . (for example: ptr_MYPOINT).
To declare a pointer you have to put an * in front of its name. A pointer can be typed or untyped. (A typed pointer points to a particular variable type such as an integer. An untyped pointer points to any data type). See the following example of a declaration of a typed pointer and an untyped pointer:
#include int main() { int x; int *ptr_p; x = 5; ptr_p = &x; return 0; } |
Put the address of an integer into a “pointer to an integer” by using the & operator (address operator)
in front of the integer, to get the integer’s address.
in front of the integer, to get the integer’s address.
Let’s take a look at an example:
Note: The integer x contains the value 5 on a specific address. The address of the integer x is copied in the pointer ptr_p. So ptr_p points to the address of x. In short: ptr_p = &x; means, “Assign to ptr_p the address of x.”
To access the value of the integer that is being pointed to, you have to dereference the pointer. The * is used to dereference a pointer. Take a look at the following example:
#include int main() { int x; int *ptr_p; x = 5; ptr_p = &x; return 0; } |
To access the value of the integer that is being pointed to, you have to dereference the pointer. The * is used to dereference a pointer. Take a look at the following example:
#include int main() { int x,y; int *ptr_p; x = 5; ptr_p = &x; y = *ptr_p; printf("%d\n", y); return 0; } |
Note: The integer x has a value of five. The pointer ptr_p gets the address of integer x.
The value pointed to is *ptr_p. (in this case five). So the integer y now contains the value of five.
The value pointed to is *ptr_p. (in this case five). So the integer y now contains the value of five.
Take a look at the following example:
#include int main() { int x; int *ptr_p; x = 5; ptr_p = &x; *ptr_p = 10; printf("%d\n", x); return 0; } |
Note: First the ptr_p contains the address of the integer x (ptr_p = &x). Then we change the value of the integer where ptr_p is pointing to (*ptr_p = 10;). The integer x now equals ten.
Arithmetic on pointers(Pointer Arithmetic)
You can perform a limited number of arithmetic operations on pointers. These operations are:
- Increment and decrement
- Addition and subtraction
- Comparison
- Assignment
The increment (++) operator increases the value of a pointer by the size of the data object the pointer refers to. For example, if the pointer refers to the second element in an array, the ++ makes the pointer refer to the third element in the array.
The decrement (--) operator decreases the value of a pointer by the size of the data object the pointer refers to. For example, if the pointer refers to the second element in an array, the -- makes the pointer refer to the first element in the array.
You can add an integer to a pointer but you cannot add a pointer to a pointer.
If the pointer p points to the first element in an array, the following expression causes the pointer to point to the third element in the same array:
p = p + 2;
If you have two pointers that point to the same array, you can subtract one pointer from the other. This operation yields the number of elements in the array that separate the two addresses that the pointers refer to.
You can compare two pointers with the following operators: ==, !=, <, >, <=, and >=.
Pointer comparisons are defined only when the pointers point to elements of the same array. Pointer comparisons using the == and != operators can be performed even when the pointers point to elements of different arrays.
You can assign to a pointer the address of a data object, the value of another compatible pointer or the NULLpointer.
Arithmetic on pointers(Pointer Arithmetic)
You can perform a limited number of arithmetic operations on pointers. These operations are:
- Increment and decrement
- Addition and subtraction
- Comparison
- Assignment
The increment (++) operator increases the value of a pointer by the size of the data object the pointer refers to. For example, if the pointer refers to the second element in an array, the ++ makes the pointer refer to the third element in the array.
The decrement (--) operator decreases the value of a pointer by the size of the data object the pointer refers to. For example, if the pointer refers to the second element in an array, the -- makes the pointer refer to the first element in the array.
You can add an integer to a pointer but you cannot add a pointer to a pointer.
If the pointer p points to the first element in an array, the following expression causes the pointer to point to the third element in the same array:
p = p + 2;
If you have two pointers that point to the same array, you can subtract one pointer from the other. This operation yields the number of elements in the array that separate the two addresses that the pointers refer to.
You can compare two pointers with the following operators: ==, !=, <, >, <=, and >=.
Pointer comparisons are defined only when the pointers point to elements of the same array. Pointer comparisons using the == and != operators can be performed even when the pointers point to elements of different arrays.
You can assign to a pointer the address of a data object, the value of another compatible pointer or the NULLpointer.
Memory Allocation
Definition - What does Memory Allocation mean?
Memory allocation is a process by which computer programs and services are assigned with physical or virtual memory space.
Memory allocation is the process of reserving a partial or complete portion of computer memory for the execution of programs and processes. Memory allocation is achieved through a process known as memory management.
Memory allocation is primarily a computer hardware operation but is managed through operating system and software applications. Memory allocation process is quite similar in physical and virtual memory management. Programs and services are assigned with a specific memory as per their requirements when they are executed. Once the program has finished its operation or is idle, the memory is released and allocated to another program or merged within the primary memory.
Memory allocation has two core types;
- Static Memory Allocation: The program is allocated memory at compile time.
- Dynamic Memory Allocation: The programs are allocated with memory at run time.
Memory Allocation
Definition - What does Memory Allocation mean?
Memory allocation is a process by which computer programs and services are assigned with physical or virtual memory space.
Memory allocation is the process of reserving a partial or complete portion of computer memory for the execution of programs and processes. Memory allocation is achieved through a process known as memory management.
Memory allocation is the process of reserving a partial or complete portion of computer memory for the execution of programs and processes. Memory allocation is achieved through a process known as memory management.
Memory allocation is primarily a computer hardware operation but is managed through operating system and software applications. Memory allocation process is quite similar in physical and virtual memory management. Programs and services are assigned with a specific memory as per their requirements when they are executed. Once the program has finished its operation or is idle, the memory is released and allocated to another program or merged within the primary memory.
Memory allocation has two core types;
Memory allocation has two core types;
- Static Memory Allocation: The program is allocated memory at compile time.
- Dynamic Memory Allocation: The programs are allocated with memory at run time.
Dangling pointers
Dangling pointers
Dangling pointers
What is dangling pointer?
Dangling pointers and wild pointers in computer programming are pointers that do not point to a valid object of the appropriate type.
In many applications memory is allocated for holding data objects. After using these objects, tha aplication will de-allocate this memory so that the memory can be re-used. In some cases the alications may use a pointer to an object whose memory is already de-allocated. This may lead to application crash or an unpredictable behavior.
scenarios which leads to dangling pointer
- Application makes use of a object after it has been released, and there by access to an invalid memory location.
- A function returns a pointer to one of its local variables, and since this local variable is defined only fro the function, the pointer becomes invalid once the function ends.
The most common result of this bug is the crash of the application or its running thread.
Examle 1:
Example 2:
Example 3:
What is dangling pointer?
What is dangling pointer?
What is dangling pointer?
Dangling pointers and wild pointers in computer programming are pointers that do not point to a valid object of the appropriate type.
In many applications memory is allocated for holding data objects. After using these objects, tha aplication will de-allocate this memory so that the memory can be re-used. In some cases the alications may use a pointer to an object whose memory is already de-allocated. This may lead to application crash or an unpredictable behavior.
scenarios which leads to dangling pointer
- Application makes use of a object after it has been released, and there by access to an invalid memory location.
- A function returns a pointer to one of its local variables, and since this local variable is defined only fro the function, the pointer becomes invalid once the function ends.
The most common result of this bug is the crash of the application or its running thread.
Examle 1:
Example 2:
Example 3:
Dangling pointers and wild pointers in computer programming are pointers that do not point to a valid object of the appropriate type.
In many applications memory is allocated for holding data objects. After using these objects, tha aplication will de-allocate this memory so that the memory can be re-used. In some cases the alications may use a pointer to an object whose memory is already de-allocated. This may lead to application crash or an unpredictable behavior.
scenarios which leads to dangling pointer
- Application makes use of a object after it has been released, and there by access to an invalid memory location.
- A function returns a pointer to one of its local variables, and since this local variable is defined only fro the function, the pointer becomes invalid once the function ends.
The most common result of this bug is the crash of the application or its running thread.
Examle 1:
Example 2:
Example 3:
Memory leaks
In computer science, a memory leak occurs when a computer program incorrectly manages memory allocations.[1] In object-oriented programming, a memory leak may happen when an object is stored in memory but cannot be accessed by the running code.[2] A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program.
Because they can exhaust available system memory as an application runs, memory leaks are often the cause of or a contributing factor to software aging.
A memory leak, in computer science (or leakage, in this context), occurs when a computer program consumes memory but is unable to release it back to the operating system. A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program source code; however, many people refer to any unwanted increase in memory usage as a memory leak, though this is not strictly accurate.
Consequences[edit]
A memory leak can diminish the performance of the computer by reducing the amount of available memory. Eventually, in the worst case, too much of the available memory may become allocated and all or part of the system or device stops working correctly, the application fails, or the system slows down unacceptably due to thrashing.
Memory leaks may not be serious or even detectable by normal means. In modern operating systems, normal memory used by an application is released when the application terminates. This means that a memory leak in a program that only runs for a short time may not be noticed and is rarely serious.
Much more serious leaks include those:
- where the program runs for an extended time and consumes additional memory over time, such as background tasks on servers, but especially in embedded devices which may be left running for many years
- where new memory is allocated frequently for one-time tasks, such as when rendering the frames of a computer game or animated video
- where the program can request memory — such as shared memory — that is not released, even when the program terminates
- where memory is very limited, such as in an embedded system or portable device
- where the leak occurs within the operating system or memory manager
- when a system device driver causes the leak
- running on an operating system that does not automatically release memory on program termination. Often on such machines if memory is lost, it can only be reclaimed by a reboot, an example of such a system being AmigaOS.[citation needed]
An example of memory leak[edit]
The following example, written in pseudocode, is intended to show how a memory leak can come about, and its effects, without needing any programming knowledge. The program in this case is part of some very simple software designed to control an elevator. This part of the program is run whenever anyone inside the elevator presses the button for a floor.
When a button is pressed:
Get some memory, which will be used to remember the floor number
Put the floor number into the memory
Are we already on the target floor?
If so, we have nothing to do: finished
Otherwise:
Wait until the lift is idle
Go to the required floor
Release the memory we used to remember the floor number
The memory leak would occur if the floor number requested is the same floor that the lift is on; the condition for releasing the memory would be skipped. Each time this case occurs, more memory is leaked.
Cases like this wouldn't usually have any immediate effects. People do not often press the button for the floor they are already on, and in any case, the lift might have enough spare memory that this could happen hundreds or thousands of times. However, the lift will eventually run out of memory. This could take months or years, so it might not be discovered despite thorough testing.
The consequences would be unpleasant; at the very least, the lift would stop responding to requests to move to another floor. If other parts of the program need memory (a part assigned to open and close the door, for example), then someone may be trapped inside, since the software cannot open the door.
The memory leak lasts until the system is reset. For example: if the lift's power were turned off the program would stop running. When power was turned on again, the program would restart and all the memory would be available again, but the slow process of memory leak would restart together with the program, eventually prejudicing the correct running of the system.
Programming issues[edit]
Memory leaks are a common error in programming, especially when using languages that have no built in automatic garbage collection, such as C and C++. Typically, a memory leak occurs because dynamically allocated memory has become unreachable. The prevalence of memory leak bugs has led to the development of a number of debugging tools to detect unreachable memory. IBM Rational Purify, BoundsChecker, Valgrind, Parasoft Insure++, Dr. Memory and memwatch are some of the more popular memory debuggers for C and C++ programs. "Conservative" garbage collection capabilities can be added to any programming language that lacks it as a built-in feature, and libraries for doing this are available for C and C++ programs. A conservative collector finds and reclaims most, but not all, unreachable memory.
Although the memory manager can recover unreachable memory, it cannot free memory that is still reachable and therefore potentially still useful. Modern memory managers therefore provide techniques for programmers to semantically mark memory with varying levels of usefulness, which correspond to varying levels of reachability. The memory manager does not free an object that is strongly reachable. An object is strongly reachable if it is reachable either directly by a strong reference or indirectly by a chain of strong references. (Astrong reference is a reference that, unlike a weak reference, prevents an object from being garbage collected.) To prevent this, the developer is responsible for cleaning up references after use, typically by setting the reference to null once it is no longer needed and, if necessary, by deregistering any event listeners that maintain strong references to the object.
In general, automatic memory management is more robust and convenient for developers, as they don't need to implement freeing routines or worry about the sequence in which cleanup is performed or be concerned about whether or not an object is still referenced. It is easier for a programmer to know when a reference is no longer needed than to know when an object is no longer referenced. However, automatic memory management can impose a performance overhead, and it does not eliminate all of the programming errors that cause memory leaks.
RAII[edit]
Main article: Resource Acquisition Is Initialization
RAII, short for Resource Acquisition Is Initialization, is an approach to the problem commonly taken in C++, D, and Ada. It involves associating scoped objects with the acquired resources, and automatically releasing the resources once the objects are out of scope. Unlike garbage collection, RAII has the advantage of knowing when objects exist and when they do not. Compare the following C and C++ examples:
/* C version */
#include <stdlib.h>
void f(int n)
{
int* array = calloc(n, sizeof(int));
do_some_work(array);
free(array);
}
// C++ version
#include <vector>
void f(int n)
{
std::vector<int> array (n);
do_some_work(array);
}
The C version, as implemented in the example, requires explicit deallocation; the array is dynamically allocated (from the heap in most C implementations), and continues to exist until explicitly freed.
The C++ version requires no explicit deallocation; it will always occur automatically as soon as the object array
goes out of scope, including if an exception is thrown. This avoids some of the overhead of garbage collection schemes. And because object destructors can free resources other than memory, RAII helps to prevent the leaking of input and output resources accessed through a handle, which mark-and-sweep garbage collection does not handle gracefully. These include open files, open windows, user notifications, objects in a graphics drawing library, thread synchronisation primitives such as critical sections, network connections, and connections to the Windows Registry or another database.
However, using RAII correctly is not always easy and has its own pitfalls. For instance, if one is not careful, it is possible to createdangling pointers (or references) by returning data by reference, only to have that data be deleted when its containing object goes out of scope.
D uses a combination of RAII and garbage collection, employing automatic destruction when it is clear that an object cannot be accessed outside its original scope, and garbage collection otherwise.
Reference counting and cyclic references[edit]
More modern garbage collection schemes are often based on a notion of reachability - if you don't have a usable reference to the memory in question, it can be collected. Other garbage collection schemes can be based on reference counting, where an object is responsible for keeping track of how many references are pointing to it. If the number goes down to zero, the object is expected to release itself and allow its memory to be reclaimed. The flaw with this model is that it doesn't cope with cyclic references, and this is why nowadays most programmers are prepared to accept the burden of the more costly mark and sweep type of systems.
Dim A, B
Set A = CreateObject("Some.Thing")
Set B = CreateObject("Some.Thing")
' At this point, the two objects each have one reference,
Set A.member = B
Set B.member = A
' Now they each have two references.
Set A = Nothing ' You could still get out of it...
Set B = Nothing ' And now you've got a memory leak!
End
In practice, this trivial example would be spotted straight away and fixed. In most real examples, the cycle of references spans more than two objects, and is more difficult to detect.
A well-known example of this kind of leak came to prominence with the rise of AJAX programming techniques in web browsers in thelapsed listener problem. JavaScript code which associated a DOM element with an event handler, and failed to remove the reference before exiting, would leak memory (AJAX web pages keep a given DOM alive for a lot longer than traditional web pages, so this leak was much more apparent).
Effects[edit]
If a program has a memory leak and its memory usage is steadily increasing, there will not usually be an immediate symptom. Every physical system has a finite amount of memory, and if the memory leak is not contained (for example, by restarting the leaking program) it will sooner or later start to cause problems.
Most modern consumer desktop operating systems have both main memory which is physically housed in RAM microchips, andsecondary storage such as a hard drive. Memory allocation is dynamic - each process gets as much memory as it requests. Activepages are transferred into main memory for fast access; inactive pages are pushed out to secondary storage to make room, as needed. When a single process starts consuming a large amount of memory, it usually occupies more and more of main memory, pushing other programs out to secondary storage - usually significantly slowing performance of the system. Even if the leaking program is terminated, it may take some time for other programs to swap back into main memory, and for performance to return to normal.
When all the memory on a system is exhausted (whether there is virtual memory or only main memory, such as on an embedded system) any attempt to allocate more memory will fail. This usually causes the program attempting to allocate the memory to terminate itself, or to generate a segmentation fault. Some programs are designed to recover from this situation (possibly by falling back on pre-reserved memory). The first program to experience the out-of-memory may or may not be the program that has the memory leak.
Some multi-tasking operating systems have special mechanisms to deal with an out-of-memory condition, such as killing processes at random (which may affect "innocent" processes), or killing the largest process in memory (which presumably is the one causing the problem). Some operating systems have a per-process memory limit, to prevent any one program from hogging all of the memory on the system. The disadvantage to this arrangement is that the operating system sometimes must be re-configured to allow proper operation of programs that legitimately require large amounts of memory, such as those dealing with graphics, video, or scientific calculations.
If the memory leak is in the kernel, the operating system itself will likely fail. Computers without sophisticated memory management, such as embedded systems, may also completely fail from a persistent memory leak.
Publicly accessible systems such as web servers or routers are prone to denial-of-service attacks if an attacker discovers a sequence of operations which can trigger a leak. Such a sequence is known as an exploit.
A "sawtooth" pattern of memory utilization may be an indicator of a memory leak if the vertical drops coincide with reboots or application restarts. Care should be taken though because garbage collection points could also cause such a pattern and would show a healthy usage of the heap.
Other memory consumers[edit]
Note that constantly increasing memory usage is not necessarily evidence of a memory leak. Some applications will store ever increasing amounts of information in memory (e.g. as a cache). If the cache can grow so large as to cause problems, this may be a programming or design error, but is not a memory leak as the information remains nominally in use. In other cases, programs may require an unreasonably large amount of memory because the programmer has assumed memory is always sufficient for a particular task; for example, a graphics file processor might start by reading the entire contents of an image file and storing it all into memory, something that is not viable where a very large image exceeds available memory.
To put it another way, a memory leak arises from a particular kind of programming error, and without access to the program code, someone seeing symptoms can only guess that there might be a memory leak. It would be better to use terms such as "constantly increasing memory use" where no such inside knowledge exists.
A simple example in C[edit]
The following C function deliberately leaks memory by losing the pointer to the allocated memory. The leak can be said to occur as soon as the pointer 'a' goes out of scope, i.e. when function_which_allocates() returns without freeing 'a'.
#include <stdlib.h>
void function_which_allocates(void) {
/* allocate an array of 45 floats */
float * a = malloc(sizeof(float) * 45);
/* additional code making use of 'a' */
/* return to main, having forgotten to free the memory we malloc'd */
}
int main(void) {
function_which_allocates();
/* the pointer 'a' no longer exists, and therefore cannot be freed,
but the memory is still allocated. a leak has occurred. */
}
Memory leaks
Memory leaks
Memory leaks
In computer science, a memory leak occurs when a computer program incorrectly manages memory allocations.[1] In object-oriented programming, a memory leak may happen when an object is stored in memory but cannot be accessed by the running code.[2] A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program.
Because they can exhaust available system memory as an application runs, memory leaks are often the cause of or a contributing factor to software aging.
A memory leak, in computer science (or leakage, in this context), occurs when a computer program consumes memory but is unable to release it back to the operating system. A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program source code; however, many people refer to any unwanted increase in memory usage as a memory leak, though this is not strictly accurate.
Consequences[edit]
A memory leak can diminish the performance of the computer by reducing the amount of available memory. Eventually, in the worst case, too much of the available memory may become allocated and all or part of the system or device stops working correctly, the application fails, or the system slows down unacceptably due to thrashing.
Memory leaks may not be serious or even detectable by normal means. In modern operating systems, normal memory used by an application is released when the application terminates. This means that a memory leak in a program that only runs for a short time may not be noticed and is rarely serious.
Much more serious leaks include those:
- where the program runs for an extended time and consumes additional memory over time, such as background tasks on servers, but especially in embedded devices which may be left running for many years
- where new memory is allocated frequently for one-time tasks, such as when rendering the frames of a computer game or animated video
- where the program can request memory — such as shared memory — that is not released, even when the program terminates
- where memory is very limited, such as in an embedded system or portable device
- where the leak occurs within the operating system or memory manager
- when a system device driver causes the leak
- running on an operating system that does not automatically release memory on program termination. Often on such machines if memory is lost, it can only be reclaimed by a reboot, an example of such a system being AmigaOS.[citation needed]
An example of memory leak[edit]
The following example, written in pseudocode, is intended to show how a memory leak can come about, and its effects, without needing any programming knowledge. The program in this case is part of some very simple software designed to control an elevator. This part of the program is run whenever anyone inside the elevator presses the button for a floor.
When a button is pressed:
Get some memory, which will be used to remember the floor number
Put the floor number into the memory
Are we already on the target floor?
If so, we have nothing to do: finished
Otherwise:
Wait until the lift is idle
Go to the required floor
Release the memory we used to remember the floor number
The memory leak would occur if the floor number requested is the same floor that the lift is on; the condition for releasing the memory would be skipped. Each time this case occurs, more memory is leaked.
Cases like this wouldn't usually have any immediate effects. People do not often press the button for the floor they are already on, and in any case, the lift might have enough spare memory that this could happen hundreds or thousands of times. However, the lift will eventually run out of memory. This could take months or years, so it might not be discovered despite thorough testing.
The consequences would be unpleasant; at the very least, the lift would stop responding to requests to move to another floor. If other parts of the program need memory (a part assigned to open and close the door, for example), then someone may be trapped inside, since the software cannot open the door.
The memory leak lasts until the system is reset. For example: if the lift's power were turned off the program would stop running. When power was turned on again, the program would restart and all the memory would be available again, but the slow process of memory leak would restart together with the program, eventually prejudicing the correct running of the system.
Programming issues[edit]
Memory leaks are a common error in programming, especially when using languages that have no built in automatic garbage collection, such as C and C++. Typically, a memory leak occurs because dynamically allocated memory has become unreachable. The prevalence of memory leak bugs has led to the development of a number of debugging tools to detect unreachable memory. IBM Rational Purify, BoundsChecker, Valgrind, Parasoft Insure++, Dr. Memory and memwatch are some of the more popular memory debuggers for C and C++ programs. "Conservative" garbage collection capabilities can be added to any programming language that lacks it as a built-in feature, and libraries for doing this are available for C and C++ programs. A conservative collector finds and reclaims most, but not all, unreachable memory.
Although the memory manager can recover unreachable memory, it cannot free memory that is still reachable and therefore potentially still useful. Modern memory managers therefore provide techniques for programmers to semantically mark memory with varying levels of usefulness, which correspond to varying levels of reachability. The memory manager does not free an object that is strongly reachable. An object is strongly reachable if it is reachable either directly by a strong reference or indirectly by a chain of strong references. (Astrong reference is a reference that, unlike a weak reference, prevents an object from being garbage collected.) To prevent this, the developer is responsible for cleaning up references after use, typically by setting the reference to null once it is no longer needed and, if necessary, by deregistering any event listeners that maintain strong references to the object.
In general, automatic memory management is more robust and convenient for developers, as they don't need to implement freeing routines or worry about the sequence in which cleanup is performed or be concerned about whether or not an object is still referenced. It is easier for a programmer to know when a reference is no longer needed than to know when an object is no longer referenced. However, automatic memory management can impose a performance overhead, and it does not eliminate all of the programming errors that cause memory leaks.
RAII[edit]
Main article: Resource Acquisition Is Initialization
RAII, short for Resource Acquisition Is Initialization, is an approach to the problem commonly taken in C++, D, and Ada. It involves associating scoped objects with the acquired resources, and automatically releasing the resources once the objects are out of scope. Unlike garbage collection, RAII has the advantage of knowing when objects exist and when they do not. Compare the following C and C++ examples:
/* C version */
#include <stdlib.h>
void f(int n)
{
int* array = calloc(n, sizeof(int));
do_some_work(array);
free(array);
}
// C++ version
#include <vector>
void f(int n)
{
std::vector<int> array (n);
do_some_work(array);
}
The C version, as implemented in the example, requires explicit deallocation; the array is dynamically allocated (from the heap in most C implementations), and continues to exist until explicitly freed.
The C++ version requires no explicit deallocation; it will always occur automatically as soon as the object array
goes out of scope, including if an exception is thrown. This avoids some of the overhead of garbage collection schemes. And because object destructors can free resources other than memory, RAII helps to prevent the leaking of input and output resources accessed through a handle, which mark-and-sweep garbage collection does not handle gracefully. These include open files, open windows, user notifications, objects in a graphics drawing library, thread synchronisation primitives such as critical sections, network connections, and connections to the Windows Registry or another database.
However, using RAII correctly is not always easy and has its own pitfalls. For instance, if one is not careful, it is possible to createdangling pointers (or references) by returning data by reference, only to have that data be deleted when its containing object goes out of scope.
D uses a combination of RAII and garbage collection, employing automatic destruction when it is clear that an object cannot be accessed outside its original scope, and garbage collection otherwise.
Reference counting and cyclic references[edit]
More modern garbage collection schemes are often based on a notion of reachability - if you don't have a usable reference to the memory in question, it can be collected. Other garbage collection schemes can be based on reference counting, where an object is responsible for keeping track of how many references are pointing to it. If the number goes down to zero, the object is expected to release itself and allow its memory to be reclaimed. The flaw with this model is that it doesn't cope with cyclic references, and this is why nowadays most programmers are prepared to accept the burden of the more costly mark and sweep type of systems.
Dim A, B
Set A = CreateObject("Some.Thing")
Set B = CreateObject("Some.Thing")
' At this point, the two objects each have one reference,
Set A.member = B
Set B.member = A
' Now they each have two references.
Set A = Nothing ' You could still get out of it...
Set B = Nothing ' And now you've got a memory leak!
End
In practice, this trivial example would be spotted straight away and fixed. In most real examples, the cycle of references spans more than two objects, and is more difficult to detect.
A well-known example of this kind of leak came to prominence with the rise of AJAX programming techniques in web browsers in thelapsed listener problem. JavaScript code which associated a DOM element with an event handler, and failed to remove the reference before exiting, would leak memory (AJAX web pages keep a given DOM alive for a lot longer than traditional web pages, so this leak was much more apparent).
Effects[edit]
If a program has a memory leak and its memory usage is steadily increasing, there will not usually be an immediate symptom. Every physical system has a finite amount of memory, and if the memory leak is not contained (for example, by restarting the leaking program) it will sooner or later start to cause problems.
Most modern consumer desktop operating systems have both main memory which is physically housed in RAM microchips, andsecondary storage such as a hard drive. Memory allocation is dynamic - each process gets as much memory as it requests. Activepages are transferred into main memory for fast access; inactive pages are pushed out to secondary storage to make room, as needed. When a single process starts consuming a large amount of memory, it usually occupies more and more of main memory, pushing other programs out to secondary storage - usually significantly slowing performance of the system. Even if the leaking program is terminated, it may take some time for other programs to swap back into main memory, and for performance to return to normal.
When all the memory on a system is exhausted (whether there is virtual memory or only main memory, such as on an embedded system) any attempt to allocate more memory will fail. This usually causes the program attempting to allocate the memory to terminate itself, or to generate a segmentation fault. Some programs are designed to recover from this situation (possibly by falling back on pre-reserved memory). The first program to experience the out-of-memory may or may not be the program that has the memory leak.
Some multi-tasking operating systems have special mechanisms to deal with an out-of-memory condition, such as killing processes at random (which may affect "innocent" processes), or killing the largest process in memory (which presumably is the one causing the problem). Some operating systems have a per-process memory limit, to prevent any one program from hogging all of the memory on the system. The disadvantage to this arrangement is that the operating system sometimes must be re-configured to allow proper operation of programs that legitimately require large amounts of memory, such as those dealing with graphics, video, or scientific calculations.
If the memory leak is in the kernel, the operating system itself will likely fail. Computers without sophisticated memory management, such as embedded systems, may also completely fail from a persistent memory leak.
Publicly accessible systems such as web servers or routers are prone to denial-of-service attacks if an attacker discovers a sequence of operations which can trigger a leak. Such a sequence is known as an exploit.
A "sawtooth" pattern of memory utilization may be an indicator of a memory leak if the vertical drops coincide with reboots or application restarts. Care should be taken though because garbage collection points could also cause such a pattern and would show a healthy usage of the heap.
Other memory consumers[edit]
Note that constantly increasing memory usage is not necessarily evidence of a memory leak. Some applications will store ever increasing amounts of information in memory (e.g. as a cache). If the cache can grow so large as to cause problems, this may be a programming or design error, but is not a memory leak as the information remains nominally in use. In other cases, programs may require an unreasonably large amount of memory because the programmer has assumed memory is always sufficient for a particular task; for example, a graphics file processor might start by reading the entire contents of an image file and storing it all into memory, something that is not viable where a very large image exceeds available memory.
To put it another way, a memory leak arises from a particular kind of programming error, and without access to the program code, someone seeing symptoms can only guess that there might be a memory leak. It would be better to use terms such as "constantly increasing memory use" where no such inside knowledge exists.
A simple example in C[edit]
The following C function deliberately leaks memory by losing the pointer to the allocated memory. The leak can be said to occur as soon as the pointer 'a' goes out of scope, i.e. when function_which_allocates() returns without freeing 'a'.
#include <stdlib.h>
void function_which_allocates(void) {
/* allocate an array of 45 floats */
float * a = malloc(sizeof(float) * 45);
/* additional code making use of 'a' */
/* return to main, having forgotten to free the memory we malloc'd */
}
int main(void) {
function_which_allocates();
/* the pointer 'a' no longer exists, and therefore cannot be freed,
but the memory is still allocated. a leak has occurred. */
}
In computer science, a memory leak occurs when a computer program incorrectly manages memory allocations.[1] In object-oriented programming, a memory leak may happen when an object is stored in memory but cannot be accessed by the running code.[2] A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program.
Because they can exhaust available system memory as an application runs, memory leaks are often the cause of or a contributing factor to software aging.
A memory leak, in computer science (or leakage, in this context), occurs when a computer program consumes memory but is unable to release it back to the operating system. A memory leak has symptoms similar to a number of other problems (see below) and generally can only be diagnosed by a programmer with access to the program source code; however, many people refer to any unwanted increase in memory usage as a memory leak, though this is not strictly accurate.
Consequences[edit]
A memory leak can diminish the performance of the computer by reducing the amount of available memory. Eventually, in the worst case, too much of the available memory may become allocated and all or part of the system or device stops working correctly, the application fails, or the system slows down unacceptably due to thrashing.
Memory leaks may not be serious or even detectable by normal means. In modern operating systems, normal memory used by an application is released when the application terminates. This means that a memory leak in a program that only runs for a short time may not be noticed and is rarely serious.
Much more serious leaks include those:
- where the program runs for an extended time and consumes additional memory over time, such as background tasks on servers, but especially in embedded devices which may be left running for many years
- where new memory is allocated frequently for one-time tasks, such as when rendering the frames of a computer game or animated video
- where the program can request memory — such as shared memory — that is not released, even when the program terminates
- where memory is very limited, such as in an embedded system or portable device
- where the leak occurs within the operating system or memory manager
- when a system device driver causes the leak
- running on an operating system that does not automatically release memory on program termination. Often on such machines if memory is lost, it can only be reclaimed by a reboot, an example of such a system being AmigaOS.[citation needed]
An example of memory leak[edit]
The following example, written in pseudocode, is intended to show how a memory leak can come about, and its effects, without needing any programming knowledge. The program in this case is part of some very simple software designed to control an elevator. This part of the program is run whenever anyone inside the elevator presses the button for a floor.
When a button is pressed:
Get some memory, which will be used to remember the floor number
Put the floor number into the memory
Are we already on the target floor?
If so, we have nothing to do: finished
Otherwise:
Wait until the lift is idle
Go to the required floor
Release the memory we used to remember the floor number
The memory leak would occur if the floor number requested is the same floor that the lift is on; the condition for releasing the memory would be skipped. Each time this case occurs, more memory is leaked.
Cases like this wouldn't usually have any immediate effects. People do not often press the button for the floor they are already on, and in any case, the lift might have enough spare memory that this could happen hundreds or thousands of times. However, the lift will eventually run out of memory. This could take months or years, so it might not be discovered despite thorough testing.
The consequences would be unpleasant; at the very least, the lift would stop responding to requests to move to another floor. If other parts of the program need memory (a part assigned to open and close the door, for example), then someone may be trapped inside, since the software cannot open the door.
The memory leak lasts until the system is reset. For example: if the lift's power were turned off the program would stop running. When power was turned on again, the program would restart and all the memory would be available again, but the slow process of memory leak would restart together with the program, eventually prejudicing the correct running of the system.
Programming issues[edit]
Memory leaks are a common error in programming, especially when using languages that have no built in automatic garbage collection, such as C and C++. Typically, a memory leak occurs because dynamically allocated memory has become unreachable. The prevalence of memory leak bugs has led to the development of a number of debugging tools to detect unreachable memory. IBM Rational Purify, BoundsChecker, Valgrind, Parasoft Insure++, Dr. Memory and memwatch are some of the more popular memory debuggers for C and C++ programs. "Conservative" garbage collection capabilities can be added to any programming language that lacks it as a built-in feature, and libraries for doing this are available for C and C++ programs. A conservative collector finds and reclaims most, but not all, unreachable memory.
Although the memory manager can recover unreachable memory, it cannot free memory that is still reachable and therefore potentially still useful. Modern memory managers therefore provide techniques for programmers to semantically mark memory with varying levels of usefulness, which correspond to varying levels of reachability. The memory manager does not free an object that is strongly reachable. An object is strongly reachable if it is reachable either directly by a strong reference or indirectly by a chain of strong references. (Astrong reference is a reference that, unlike a weak reference, prevents an object from being garbage collected.) To prevent this, the developer is responsible for cleaning up references after use, typically by setting the reference to null once it is no longer needed and, if necessary, by deregistering any event listeners that maintain strong references to the object.
In general, automatic memory management is more robust and convenient for developers, as they don't need to implement freeing routines or worry about the sequence in which cleanup is performed or be concerned about whether or not an object is still referenced. It is easier for a programmer to know when a reference is no longer needed than to know when an object is no longer referenced. However, automatic memory management can impose a performance overhead, and it does not eliminate all of the programming errors that cause memory leaks.
RAII[edit]
Main article: Resource Acquisition Is Initialization
RAII, short for Resource Acquisition Is Initialization, is an approach to the problem commonly taken in C++, D, and Ada. It involves associating scoped objects with the acquired resources, and automatically releasing the resources once the objects are out of scope. Unlike garbage collection, RAII has the advantage of knowing when objects exist and when they do not. Compare the following C and C++ examples:
/* C version */ #include <stdlib.h> void f(int n) { int* array = calloc(n, sizeof(int)); do_some_work(array); free(array); }
// C++ version #include <vector> void f(int n) { std::vector<int> array (n); do_some_work(array); }
The C version, as implemented in the example, requires explicit deallocation; the array is dynamically allocated (from the heap in most C implementations), and continues to exist until explicitly freed.
The C++ version requires no explicit deallocation; it will always occur automatically as soon as the object
array
goes out of scope, including if an exception is thrown. This avoids some of the overhead of garbage collection schemes. And because object destructors can free resources other than memory, RAII helps to prevent the leaking of input and output resources accessed through a handle, which mark-and-sweep garbage collection does not handle gracefully. These include open files, open windows, user notifications, objects in a graphics drawing library, thread synchronisation primitives such as critical sections, network connections, and connections to the Windows Registry or another database.
However, using RAII correctly is not always easy and has its own pitfalls. For instance, if one is not careful, it is possible to createdangling pointers (or references) by returning data by reference, only to have that data be deleted when its containing object goes out of scope.
D uses a combination of RAII and garbage collection, employing automatic destruction when it is clear that an object cannot be accessed outside its original scope, and garbage collection otherwise.
Reference counting and cyclic references[edit]
More modern garbage collection schemes are often based on a notion of reachability - if you don't have a usable reference to the memory in question, it can be collected. Other garbage collection schemes can be based on reference counting, where an object is responsible for keeping track of how many references are pointing to it. If the number goes down to zero, the object is expected to release itself and allow its memory to be reclaimed. The flaw with this model is that it doesn't cope with cyclic references, and this is why nowadays most programmers are prepared to accept the burden of the more costly mark and sweep type of systems.
Dim A, B Set A = CreateObject("Some.Thing") Set B = CreateObject("Some.Thing") ' At this point, the two objects each have one reference, Set A.member = B Set B.member = A ' Now they each have two references. Set A = Nothing ' You could still get out of it... Set B = Nothing ' And now you've got a memory leak! End
In practice, this trivial example would be spotted straight away and fixed. In most real examples, the cycle of references spans more than two objects, and is more difficult to detect.
A well-known example of this kind of leak came to prominence with the rise of AJAX programming techniques in web browsers in thelapsed listener problem. JavaScript code which associated a DOM element with an event handler, and failed to remove the reference before exiting, would leak memory (AJAX web pages keep a given DOM alive for a lot longer than traditional web pages, so this leak was much more apparent).
Effects[edit]
If a program has a memory leak and its memory usage is steadily increasing, there will not usually be an immediate symptom. Every physical system has a finite amount of memory, and if the memory leak is not contained (for example, by restarting the leaking program) it will sooner or later start to cause problems.
Most modern consumer desktop operating systems have both main memory which is physically housed in RAM microchips, andsecondary storage such as a hard drive. Memory allocation is dynamic - each process gets as much memory as it requests. Activepages are transferred into main memory for fast access; inactive pages are pushed out to secondary storage to make room, as needed. When a single process starts consuming a large amount of memory, it usually occupies more and more of main memory, pushing other programs out to secondary storage - usually significantly slowing performance of the system. Even if the leaking program is terminated, it may take some time for other programs to swap back into main memory, and for performance to return to normal.
When all the memory on a system is exhausted (whether there is virtual memory or only main memory, such as on an embedded system) any attempt to allocate more memory will fail. This usually causes the program attempting to allocate the memory to terminate itself, or to generate a segmentation fault. Some programs are designed to recover from this situation (possibly by falling back on pre-reserved memory). The first program to experience the out-of-memory may or may not be the program that has the memory leak.
Some multi-tasking operating systems have special mechanisms to deal with an out-of-memory condition, such as killing processes at random (which may affect "innocent" processes), or killing the largest process in memory (which presumably is the one causing the problem). Some operating systems have a per-process memory limit, to prevent any one program from hogging all of the memory on the system. The disadvantage to this arrangement is that the operating system sometimes must be re-configured to allow proper operation of programs that legitimately require large amounts of memory, such as those dealing with graphics, video, or scientific calculations.
If the memory leak is in the kernel, the operating system itself will likely fail. Computers without sophisticated memory management, such as embedded systems, may also completely fail from a persistent memory leak.
Publicly accessible systems such as web servers or routers are prone to denial-of-service attacks if an attacker discovers a sequence of operations which can trigger a leak. Such a sequence is known as an exploit.
A "sawtooth" pattern of memory utilization may be an indicator of a memory leak if the vertical drops coincide with reboots or application restarts. Care should be taken though because garbage collection points could also cause such a pattern and would show a healthy usage of the heap.
Other memory consumers[edit]
Note that constantly increasing memory usage is not necessarily evidence of a memory leak. Some applications will store ever increasing amounts of information in memory (e.g. as a cache). If the cache can grow so large as to cause problems, this may be a programming or design error, but is not a memory leak as the information remains nominally in use. In other cases, programs may require an unreasonably large amount of memory because the programmer has assumed memory is always sufficient for a particular task; for example, a graphics file processor might start by reading the entire contents of an image file and storing it all into memory, something that is not viable where a very large image exceeds available memory.
To put it another way, a memory leak arises from a particular kind of programming error, and without access to the program code, someone seeing symptoms can only guess that there might be a memory leak. It would be better to use terms such as "constantly increasing memory use" where no such inside knowledge exists.
A simple example in C[edit]
The following C function deliberately leaks memory by losing the pointer to the allocated memory. The leak can be said to occur as soon as the pointer 'a' goes out of scope, i.e. when function_which_allocates() returns without freeing 'a'.
#include <stdlib.h> void function_which_allocates(void) { /* allocate an array of 45 floats */ float * a = malloc(sizeof(float) * 45); /* additional code making use of 'a' */ /* return to main, having forgotten to free the memory we malloc'd */ } int main(void) { function_which_allocates(); /* the pointer 'a' no longer exists, and therefore cannot be freed, but the memory is still allocated. a leak has occurred. */ }