In the history of AI development, one of the greatest AI technologies was the pattern recognition models, especially to read texts from pixel images. One of them was the Hopfield network (or Ising model of a neural network or Ising–Lenz–Little model), which is a form of recurrent neural network, it was invented by Dr. John J. Hopfield in 1982. A Hopfield Network can be applied to pattern recognition, such as recognition from digit character images. In this post, we will develop a simple Hopfield Network example in GUI form using C++ Builder that can learn from pixelated patterns and we can recall them by testing some closest patterns.
Table of Contents
What is Hopfield Network in AI development?
The purpose of a Hopfield Network is to store data patterns and to recall the full data pattern based on partial input. For example, a Hopfield Network can be applied to 8×8 digit character recognition from pixels. We can train some characters on this network and then we can ask a closer drawing character if it remembers trained one or not. We can use this Hopfield network to extract the characters from images and put them in a text file in ASCII form, which is called pattern recognition. The good behavior of this network is it can also remember the full form of that character while it is not given completely. This example can be used for the optical character recognition from a text and if there is deformation on a small part of a letter or on a paper, maybe be paper is dirty Hopefield network can remember this kind of problem. In new modern ML and AI applications, there are much more useful methods that are based on this Hopfield Network like Recurrent Artificial Neural Networks.
The Hopfield Network (or Ising model of a neural network or Ising–Lenz–Little model) is a form of recurrent neural network, and it was invented by Dr. John J. Hopfield in 1982. It consists of a single layer that contains one or more fully connected recurrent neurons. The Hopfield network is commonly used for auto-association and optimization tasks. Hopfield networks serve as content-addressable memory systems with binary threshold nodes. Hopfield networks also provide a primitive model for understanding how the human brain and memory can be simulated artificially.
How to Train a Hopfield Network
How we can apply this method on a digital character recognition in programming ? There is a way a Hopfield network would work on 1 and 0 pixels. We can map it out that each pixel represents one node in the Hopfield network. We can train correct form of the characters on this network to recognize each of characters. Hopfield network finds the most possible assumed character after few iterations, and eventually reproduces the pattern with the trained correct form. If we have N pixels, it means our network has NxN weights, so the problem is very computationally expensive and may be slow in some cases. In example, we can train blank form (space) and this “A” form, and so on.
All the nodes in a Hopfield network are used as both inputs and outputs, and they are fully interconnected with each other. That is, each node is an input to every other node in the network. We can think of the links from each node to itself as being a link with a weight of 0.
We can easily train Hopfield networks, we just need binary data to train this network. As we know we can have binary input vectors as well as bipolar input vectors. During the training of the Hopfield network, weights are being updated in iterations.
For example for a 3 node Hopefield network data matrix a composed with N elements and weight matrix composed with NxN elements. Here is a 3 node example to these matrixes,
How can we develop a simple Hopfield Network in C++?
Let’s create a simple Hofield Network C++ Builder example as below.
We can create a simper THopfield_Network class as in given steps below. Here is how we start to define out class,
|
1 2 3 4 5 6 7 8 9 10 11 12 |
class THopfield_Network { private: std::vector<std::vector<int>> weights; public: THopfield_Network(int numofNeurons) : weights(numofNeurons, std::vector<int>(numofNeurons, 0)) { } } |
We will add 3 public methods to this class. First, it will learn from a pattern vector by using learn_pattern() method which is defined as below,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
// Learn from a pattern (update weights) void learn_pattern(const std::vector<int>& pattern) { for (int i = 0; i < pattern.size(); ++i) { for (int j = 0; j < pattern.size(); ++j) { if (i != j) { weights[i][j] += pattern[i] * pattern[j]; } } } } |
And we will update neurons of our hopfield network by using update_neuron() method which is defined as below,
|
1 2 3 4 5 6 7 8 9 10 11 12 |
// Update neuron asynchronously int update_neuron(const std::vector<int>& input, int neuronIndex) { int sum = 0; for (int i = 0; i < input.size(); ++i) { sum += weights[neuronIndex][i] * input[i]; } return (sum >= 0) ? 1 : -1; } |
We can use this update_neuron method to test given input pattern by using a test() method which is defined as below,
|
1 2 3 4 5 6 7 8 9 10 11 12 |
// Test the network std::vector<int> test(const std::vector<int>& input) { std::vector<int> output(input); for (int i = 0; i < input.size(); ++i) { output[i] = update_neuron(output, i); } return output; } |
As a result our simple Hopfield Network class in modern C++ will be as below,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
class THopfield_Network { private: std::vector<std::vector<int>> weights; public: THopfield_Network(int numofNeurons) : weights(numofNeurons, std::vector<int>(numofNeurons, 0)) { } // Learn from a pattern (update weights) void learn_pattern(const std::vector<int>& pattern) { for (int i = 0; i < pattern.size(); ++i) { for (int j = 0; j < pattern.size(); ++j) { if (i != j) { weights[i][j] += pattern[i] * pattern[j]; } } } } // Update neuron asynchronously int update_neuron(const std::vector<int>& input, int neuronIndex) { int sum = 0; for (int i = 0; i < input.size(); ++i) { sum += weights[neuronIndex][i] * input[i]; } return (sum >= 0) ? 1 : -1; } // Test the network std::vector<int> test(const std::vector<int>& input) { std::vector<int> output(input); for (int i = 0; i < input.size(); ++i) { output[i] = update_neuron(output, i); } return output; } }; |
Now we can globally define a hopfield network for given Width and Height pixels. In example if we have 6×6 pixels patterns we can define it as in below,
|
1 2 3 4 5 |
int W = 6, H = 6; THopfield_Network hopfield(W*H); |
Now we can define a pattern vector as below,
|
1 2 3 4 5 6 7 8 9 10 |
std::vector<int> pattern = { 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1, -1, -1, -1, -1, -1, -1, 1 }; |
we can train this pattern and if we have another input pattern as above, we can test it to obtain closest result (recovered_pattern) to given input as below.
|
1 2 3 4 5 |
hopfield.learn_pattern(pattern); std::vector<int> recovered_pattern = hopfield.test(input_pattern); |
Now, let’s do all this with GUIs in C++ Builder.
How can we develop a simple Hopfield Network in C++ Builder?
- First, create a new C++ Builder FMX application, add an Image (TImage), Memo (TMemo) and 3 Buttons (TButton) which are “Clear”, “Train” and “Test” buttons. You can add some Layouts to arrange them as given Form (TForm) design below.
2. Add our THopfield_Network class to below the line “TForm1 *Form1;”,
3. Define bmp and bmp2 bitmaps as in this Form unit’s header file as below, bmp will be used as an input pattern. bmp2 will be displayed bitmap on Image.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
class TForm1 : public TForm { __published: // IDE-managed Components TImage *Image1; TButton *btTrain; TButton *btTest; TButton *btClear; TMemo *Memo1; TLayout *Layout1; TLayout *Layout2; void __fastcall btClearClick(TObject *Sender); void __fastcall Image1MouseDown(TObject *Sender, TMouseButton Button, TShiftState Shift, float X, float Y); void __fastcall btTrainClick(TObject *Sender); void __fastcall btTestClick(TObject *Sender); private: // User declarations public: // User declarations TBitmap *bmp, *bmp2; __fastcall TForm1(TComponent* Owner); }; |
4. Now we can create these bitmaps when the Form is constructed as below.
|
1 2 3 4 5 6 7 8 9 |
__fastcall TForm1::TForm1(TComponent* Owner) : TForm(Owner) { bmp = new TBitmap(W,H); bmp2 = new TBitmap(Image1->Width, Image1->Height); Image1->Bitmap->Assign(bmp2); } |
5. Now we can create clear event by double clicking to Clear button, such as,
|
1 2 3 4 5 6 7 |
void __fastcall TForm1::btClearClick(TObject *Sender) { bmp->Clear(claBlack); Image1->Bitmap->Clear(claBlack); } |
6. We should allow user to define its own pattern by clicking on the Image, so user can draw some patterns on our Image1. We can create this by double clicking OnMouseDown event as below,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
void __fastcall TForm1::Image1MouseDown(TObject *Sender, TMouseButton Button, TShiftState Shift, float X, float Y) { float w = Image1->Width/W; // grid width float h = Image1->Height/H; // grid height int px = X/w; // exact pixel X on the bitmap of scaled image int py = Y/h; // exact pixel Y on the bitmap of scaled image TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { acr.Color = bitmapData.GetPixel( px, py ); if(acr.Color == claBlack) { bitmapData.SetPixel(px,py, claWhite); Image1->Bitmap->Canvas->BeginScene(); Image1->Bitmap->Canvas->Fill->Color = claWhite; Image1->Bitmap->Canvas->FillRect( TRectF(px*w,py*h, px*w+w,py*h+h), 0, 0, AllCorners, 255.0); Image1->Bitmap->Canvas->EndScene(); } else { bitmapData.SetPixel(px,py, claBlack); Image1->Bitmap->Canvas->BeginScene(); Image1->Bitmap->Canvas->Fill->Color = claBlack; Image1->Bitmap->Canvas->FillRect( TRectF(px*w,py*h, px*w+w,py*h+h), 0, 0, AllCorners, 255.0); Image1->Bitmap->Canvas->EndScene(); } bmp->Unmap(bitmapData); } } |
7. Now, we can create training method by double clicking our Train button. In this method, we will read data from pixels of bmp bitmap then we will display this in Memo box, and then we will learn this pattern by using hopfield.learn_pattern(pattern); Here is how we do this below,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
void __fastcall TForm1::btTrainClick(TObject *Sender) { std::vector<int> pattern; TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { for(int j=0; j<H; j++) for(int i=0; i<W; i++) { acr.Color = bitmapData.GetPixel( i, j ); if(acr.Color == claBlack) pattern.push_back(-1); else pattern.push_back(1); } bmp->Unmap(bitmapData); } Memo1->Lines->Add("Learned Pattern:"); for(int j=0; j<H; j++) { String str =""; for(int i=0; i<W; i++) { if(pattern[j*W+i]==1) str += "* "; else str += " "; } Memo1->Lines->Add(str); } Memo1->Lines->Add("------------"); hopfield.learn_pattern(pattern); } |
8. Finally we can create Testing event by double clicking to Test button.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
void __fastcall TForm1::btTestClick(TObject *Sender) { std::vector<int> pattern; TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { for(int j=0; j<H; j++) for(int i=0; i<W; i++) { acr.Color = bitmapData.GetPixel( i, j ); if(acr.Color == claBlack) pattern.push_back(-1); else pattern.push_back(1); } bmp->Unmap(bitmapData); } std::vector<int> recoveredpattern = hopfield.test(pattern); float w = Image1->Width/W; // grid width float h = Image1->Height/H; // grid height Memo1->Lines->Add("Recovered Pattern:"); Image1->Bitmap->Canvas->BeginScene(); for(int j=0; j<H; j++) { String str =""; for(int i=0; i<W; i++) { if(recoveredpattern[j*W+i]==1) { Image1->Bitmap->Canvas->Fill->Color = claWhite; Image1->Bitmap->Canvas->FillRect( TRectF(i*w, j*h, i*w+w, j*h+h), 0, 0, AllCorners, 255.0); str += "* "; } else { Image1->Bitmap->Canvas->Fill->Color = claBlack; Image1->Bitmap->Canvas->FillRect( TRectF(i*w, j*h, i*w+w, j*h+h), 0, 0, AllCorners, 255.0); str += " "; } } Memo1->Lines->Add(str); } Image1->Bitmap->Canvas->EndScene(); Memo1->Lines->Add("------------"); } |
How can we test a simple Hopfield Network app in C++ Builder?
Now we can run our application (F9). First, we can Clear and then we Train this blank pattern, then we can draw “A” as below and then we can Train this. You can train more different patterns.
After this step, we can ask another pattern by using Test button that looks like one of the pattern that we trained before, such as this,
As a result you will see that our Hopfield Network remembers its closest pattern and recovers it.
As you see developing AI technologies in C++ Builder is really easy and amazing to understand AI technologies that build helps to build today’s models.
Is there a full example C++ code about Hopfield Network in C++ Builder?
Here is the full example of C++ Builder FMX application, note that header is also given above,
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 |
//--------------------------------------------------------------------------- #include <fmx.h> #include <vector> #pragma hdrstop #include "Hopfield_Network_FMX_Unit1.h" //--------------------------------------------------------------------------- #pragma package(smart_init) #pragma resource "*.fmx" TForm1 *Form1; class THopfield_Network { private: std::vector<std::vector<int>> weights; public: THopfield_Network(int numofNeurons) : weights(numofNeurons, std::vector<int>(numofNeurons, 0)) { } // Learn from a pattern (update weights) void learn_pattern(const std::vector<int>& pattern) { for (int i = 0; i < pattern.size(); ++i) { for (int j = 0; j < pattern.size(); ++j) { if (i != j) { weights[i][j] += pattern[i] * pattern[j]; } } } } // Update neuron asynchronously int update_neuron(const std::vector<int>& input, int neuronIndex) { int sum = 0; for (int i = 0; i < input.size(); ++i) { sum += weights[neuronIndex][i] * input[i]; } return (sum >= 0) ? 1 : -1; } // Test the network std::vector<int> test(const std::vector<int>& input) { std::vector<int> output(input); for (int i = 0; i < input.size(); ++i) { output[i] = update_neuron(output, i); } return output; } }; int W = 6, H = 6; THopfield_Network hopfield(W*H); //--------------------------------------------------------------------------- __fastcall TForm1::TForm1(TComponent* Owner) : TForm(Owner) { bmp = new TBitmap(W,H); bmp2 = new TBitmap(Image1->Width, Image1->Height); Image1->Bitmap->Assign(bmp2); } //--------------------------------------------------------------------------- void __fastcall TForm1::btClearClick(TObject *Sender) { bmp->Clear(claBlack); Image1->Bitmap->Clear(claBlack); } //--------------------------------------------------------------------------- void __fastcall TForm1::Image1MouseDown(TObject *Sender, TMouseButton Button, TShiftState Shift, float X, float Y) { float w = Image1->Width/W; // grid width float h = Image1->Height/H; // grid height int px = X/w; // exact pixel X on the bitmap of scaled image int py = Y/h; // exact pixel Y on the bitmap of scaled image TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { acr.Color = bitmapData.GetPixel( px, py ); if(acr.Color == claBlack) { bitmapData.SetPixel(px,py, claWhite); Image1->Bitmap->Canvas->BeginScene(); Image1->Bitmap->Canvas->Fill->Color = claWhite; Image1->Bitmap->Canvas->FillRect( TRectF(px*w,py*h, px*w+w,py*h+h), 0, 0, AllCorners, 255.0); Image1->Bitmap->Canvas->EndScene(); } else { bitmapData.SetPixel(px,py, claBlack); Image1->Bitmap->Canvas->BeginScene(); Image1->Bitmap->Canvas->Fill->Color = claBlack; Image1->Bitmap->Canvas->FillRect( TRectF(px*w,py*h, px*w+w,py*h+h), 0, 0, AllCorners, 255.0); Image1->Bitmap->Canvas->EndScene(); } bmp->Unmap(bitmapData); } } //--------------------------------------------------------------------------- void __fastcall TForm1::btTrainClick(TObject *Sender) { std::vector<int> pattern; TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { for(int j=0; j<H; j++) for(int i=0; i<W; i++) { acr.Color = bitmapData.GetPixel( i, j ); if(acr.Color == claBlack) pattern.push_back(-1); else pattern.push_back(1); } bmp->Unmap(bitmapData); } Memo1->Lines->Add("Learned Pattern:"); for(int j=0; j<H; j++) { String str =""; for(int i=0; i<W; i++) { if(pattern[j*W+i]==1) str += "* "; else str += " "; } Memo1->Lines->Add(str); } Memo1->Lines->Add("------------"); hopfield.learn_pattern(pattern); } //--------------------------------------------------------------------------- void __fastcall TForm1::btTestClick(TObject *Sender) { std::vector<int> pattern; TBitmapData bitmapData; TAlphaColorRec acr; if( bmp->Map(TMapAccess::ReadWrite, bitmapData)) // Lock bitmap and retrive bitmap data { for(int j=0; j<H; j++) for(int i=0; i<W; i++) { acr.Color = bitmapData.GetPixel( i, j ); if(acr.Color == claBlack) pattern.push_back(-1); else pattern.push_back(1); } bmp->Unmap(bitmapData); } std::vector<int> recoveredpattern = hopfield.test(pattern); float w = Image1->Width/W; // grid width float h = Image1->Height/H; // grid height Memo1->Lines->Add("Recovered Pattern:"); Image1->Bitmap->Canvas->BeginScene(); for(int j=0; j<H; j++) { String str =""; for(int i=0; i<W; i++) { if(recoveredpattern[j*W+i]==1) { Image1->Bitmap->Canvas->Fill->Color = claWhite; Image1->Bitmap->Canvas->FillRect( TRectF(i*w, j*h, i*w+w, j*h+h), 0, 0, AllCorners, 255.0); str += "* "; } else { Image1->Bitmap->Canvas->Fill->Color = claBlack; Image1->Bitmap->Canvas->FillRect( TRectF(i*w, j*h, i*w+w, j*h+h), 0, 0, AllCorners, 255.0); str += " "; } } Memo1->Lines->Add(str); } Image1->Bitmap->Canvas->EndScene(); Memo1->Lines->Add("------------"); } //--------------------------------------------------------------------------- void __fastcall TForm1::FormClose(TObject *Sender, TCloseAction &Action) { bmp->Free(); bmp2->Free(); } //--------------------------------------------------------------------------- |
C++ Builder is the easiest and fastest C and C++ IDE for building simple or professional applications on the Windows, MacOS, iOS & Android operating systems. It is also easy for beginners to learn with its wide range of samples, tutorials, help files, and LSP support for code. RAD Studio’s C++ Builder version comes with the award-winning VCL framework for high-performance native Windows apps and the powerful FireMonkey (FMX) framework for cross-platform UIs.
There is a free C++ Builder Community Edition for students, beginners, and startups; it can be downloaded from here. For professional developers, there are Professional, Architect, or Enterprise version.
