jay's old blog

this blog will be deleted soon - please visit my new blog - https://thesanguinetechtrainer.com

Entering the world of web API and related stuff

Ever since this blog started, I have written what I believe are useful tutorials on android, web development and windows development. I must have written more than 150 blog posts in total (or may be more, I don't have an exact count). One common thread across all these tutorials is that they are all geared towards application development.

That means, unless you already have an external source of data (or your application connects to a database), the tutorials above will not help you much. Of course, in most real world, work type scenarios, you can be fairly sure that the blogs above will help you with the application development which is what is expected of you. However, what if you want to change teams? What if you also wish to build a data service to complement your application development? Further, perhaps, you don't wish to change teams but play on both teams. You wish to build a data service and also the application that consumes it. If you could do that, well, that would be incredibly cool.

That is what I plan to blog about going forward. I plan to build my own data service (for a project that I hope to start next year), and obviously I will blog about it. Of course, this year, I have taken up novel writing which is taking up significant amount of my writing time. So, the blog posts will reduce in quantity, at least till the end of the year, but I will blog as often as I can.

Update 1 : I will be using the information available at the official docs at asp dot net site.

Files Directories – object or static operations

When you are working with directories and files, you are provided with two options at least. You can use the static class or you can use regular classes. To add, at this point, static class means, you cannot really instantiate objects from it. It also means, you can simply use the methods of the static class directly.

Personally, I think using the static methods makes for faster execution. It also improves the file size, that is reduces the number of lines since you can do file operations by using the static class directly. So, that is when you would use the static class. When you wish to do some quick file operations, like one time opening of a file, writing into it, and then closing it. Stuff like that.

When you use the non-static methods, you will have to instantiate the object, and then do file operations. This is useful, when you are playing around with multiple files, like hundreds at the same. In this scenario, each file or directory operation will have an object connected to it. That way, when you want to do some mid operation manipulation of a directory or file, you are free to do so.

Is that explanation good enough? I would think so, but as with many things, these are developer preferences. Even for one time operations, I prefer to use the non-static classes. You may choose the other way around. Eventually, as with life itself, these decisions are yours to make.

Building a web app – Part 2 of X

In my previous post, I talked about building a simple project. So, this is where I actually build it. Each part title is linked to particular commit. Each commit will have the name identical to the blog post that you are reading it.

The actual project is available at this link - https://bitbucket.org/studynildana/demo_asp_dot_net_1

All the work that is done in this particular blog post, will be available in the commit called ‘Part 2 of X’

In this part, the very first thing I did was to go ahead and create a new project. In Visual Studio, I chose the ‘Web’ portion in the very first portion that comes up. Then, I chose the part says ‘web’ again, and here I selected MVC. Further, I check marked the option called ‘individual accounts’ as I will be building some kind of an authentication system into this web app. Further, I unchecked ‘application insights’. I also unchecked ‘deploy to azure’. Both of these unchecked steps are something that are useful but they will be stories for another day.

Once the project was created (depending on your computer speed, this may take 30 seconds to 3 minutes), I built it to make sure that there are no errors. If there are any errors, then you should fix your visual studio installation because, this new project creation should ‘simply work’. If there are no errors, run the app and you should have the website running in a browser. I personally use Firefox but you are free to use any browser of your choice. Visual Studio by default will try to run it in Edge.

Once the web app is running, I want to check if the database is working. You know, the default website SQL server express that is part of this thing. So, I went ahead and registered a dummy account. Once the registration is complete, then I logged out and logged back in. Doing these steps is essential because this tells you that the app has proper access the local database. If you run into database errors, then you should check your SQL server express installation and fix that first.

I did not run into any errors. So, at this point, I have a fully functional but empty web app running.

In the next part, I will make some UI changes to get things kicking.

Issues you might face

While pushing the project to the bit bucket, I ran into this issue where the repo end would hang up on me unexpectedly. This probably happened because I was pushing too many files and of large sizes (my guess) because the error does not come when the files being pushed are small size. I had to go with this command to get the error to go away.

Error - fatal: The remote end hung up unexpectedly

Solution - git config http.postBuffer 524288000

Building a web app – Part 4 of X

In my previous post, I talked about making some essential UI changes. Here let’s look at adding some models.

The actual project is available at this link - https://bitbucket.org/studynildana/demo_asp_dot_net_1

All the work that is done in this particular blog post, will be available in the commit called ‘Part 4 of X’

This is where the main crux of the project starts. The idea behind this mini project is to allow students to store their marks in the database. Here are the elements, that I believe that a standard student should have in our web app.

·         Name (obviously)

·         Age (of course)

·         Email address (this will be the same email address that the student will register/login to see her/his own marks)

·         Marks (I am assuming that we just have one marks and nothing else in this school. It’s not reality but so is the rainbow)

With the above data in mind, I will now add a ‘student’ model into the project.

    public class student


        //this is the unique ID automatically set by EF

        public int ID { get; set; }


        //the name of the student

        public string student_name { get; set; }


        //the age of the student

        public int student_age { get; set; }


        //email of the student

        //note that the student has to use this exact same email address when she/he registers on the web app to see her marks

        //since we cannot process login/registration process without email address, we cannot allow student entries

        //to be made to the database unless

        //so this becomes a mandatory field.

        //that is why I am using the Required attribute


        public string student_email_address { get; set; }


        //marks scored by the student

        public string student_marks { get; set; }


For the above data model, I will need to add a database set.

    //now need to get the context that will work with the database

    public class studentDbContext : DbContext


        //connection string to be used

        public studentDbContext() : base("studentDbContext")





        //DbSets that are the tables

        //table for students

        public DbSet<student> student_sets { get; set; }


        //this is more about appearance rather than functionality

        //prevents table names from being pluralized

        protected override void OnModelCreating(DbModelBuilder modelBuilder)




Now that I have the data model, and the database set, I can now use the cascading feature of visual studio and build the basic CRUD stuff for students.

Before I can do that, I will have to include a connection string. For that, I use a simple string. Remember back when I ‘checked’ individual accounts. When you did that ran the application, a connection string was already included in the web.config file of the project. So, here is how the automatically generated connection string looks like.

<add name="DefaultConnection" connectionString="Data Source=(LocalDb)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\aspnet-demo_asp_dot_net_1-20160928033935.mdf;Initial Catalog=aspnet-demo_asp_dot_net_1-20160928033935;Integrated Security=True"      providerName="System.Data.SqlClient" />

In the above data set for students, you will notice that the context is defined as studentDbContext.

<add name="studentDbContext"  connectionString="Data Source=(LocalDb)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\aspnet-demo_asp_dot_net_1-20160928033935.mdf;Initial Catalog=aspnet-demo_asp_dot_net_1-20160928033936;Integrated Security=True"      providerName="System.Data.SqlClient" />

I made changes in getting the above connection string to store the student tables (just to note, the first connection string relates to the storage of user accounts), the first being change the name. The second was change the Initial Catalog name to something else. Now, I can do the cascading with the controller and get the CRUD views for the student DB.

In the next blog post, I will use the identity management system, and also the Seed method (available from the entity framework system) to build a default set of roles as well as users.

Building a web app – Part 3 of X

In my previous blog post, I discussed creating a brand new project.

The actual project is available at this link - https://bitbucket.org/studynildana/demo_asp_dot_net_1

All the work that is done in this particular blog post, will be available in the commit called ‘Part 3 of X’

In this post, I will write about some UI changes that I did to make the web app look more like the project we are building. At this point, the project shows the standard ASP DOT NET things. For instance, the copyright marking says ‘ © 2016 - My ASP.NET Application’ Like this, there are so many ‘generic things’. I will have to change those things.

My first step is to go to the Views > Shared >_Layout.cshtml. This file is where the entire web app gets its cool menus (like when you reduce the width of the web app, the menu suddenly appears) and this is also where we can edit all the menu titles and other stuff. Best to check the commit details to see all the things I have changed.

At the end of the post, some stuff about the app (mostly visuals) have been fixed. Next post, I will talk about adding some models.

Identity Management in asp dot net

There are many reasons why web sites are now called web apps. One of those, would be, the fact that they provide at least some kind of customization based on the simple fact of a user being logged in. For instance, a website that simple shows news would be a ‘web site’. However, a web site that allows a user to login, and hence customize the news that is displayed, based on the logged in user’s preferences would become a web app.

That brings us to another essential facet of asp dot net, the identity management system. Before we understand the identity management system provided by asp, perhaps we should figure out what makes an ‘identity management system’. At its most basic form, this system should provide

·         An interface that allows new users to register.

·         An interface that allows registered users to login.

·         A system that allows these users to be assigned different roles. These roles will then be used to provide differentiated access to different parts of the web app.

·         The system should also seamlessly integrate with the database context, hiding the database related operates related to these users as well as their roles.

When building any system that operates with multiple users, the developer is tasked with building a user management system which at least offers the above. Now, it is fairly obvious that user management is a huge responsibility. One can read endless stories on the internet about how one company or the other company has been hacked and user data stolen. A discussion about the consequences of losing customer data for a company is beyond the scope of this blog post, but it is extremely bad. So, building a user management system is almost as complicated as building the rest of actual web app.

That is where asp dot net’s identity management system comes into the picture. It provides an out of the box solution that can be readily added to the web app. It provides all the essentials that I have described above, and much more (like integration with social logins such as facebook and twitter, OTP and email based password reset).

The best way to include identity management in your web app would be to ‘enable authentication’ when creating a new project. If you have already created the project, you can add the identity library manually through the package manager, but I really think that identity management should be part of your project from the project creation time. So, if you forgot to include authentication, it would simply be better to create a new project, after having looked at the mirror to realize what a moron you are.

CRUD in asp dot net

You have already learnt about CRUD in an earlier blog post. Read about it here. In this post, I am going to shed some light on the CRUD implementation in asp dot net.

The essential concept of CRUD (which is to create, read, update and delete stuff) is maintained in asp dot net. The extra coolness here (rather the coolness offered by visual studio) is the scaffolding magic that happens.

You see, when you are building CRUD around your model, you can take two routes.

·         The first route is that you manually do the View (with all the HTML and Razor code) as well as the Controller (with all the linking to the View and necessary database context c sharp code) yourself.

·         The second route is to allow the scaffolding to do its magic and add a standard CRUD mechanism into your web app. All you need to do is provide the database context and the model, and that’s it.

Personally, I like to go through the second option because, it relieves of having to do everything that is necessary to get the CRUD functionality into the system. This saves me a lot of time. Further, it also allows me to leverage on all those countless man hours Microsoft spent on getting this CRUD code right. In addition to this, I can freely modify (and of course, by the project is ready to deploy a lot of this default code will be changed to suit our own unique requirements) both the controller and view, with the explicit knowledge that the system won’t break down.

Getting this CRUD scaffolding to work is pretty straight forward. Right click on the ‘Models’ folder in solution explorer. Go to ‘Add’ and then navigate to ‘Controller’. Then, Select from the options, the one that says ‘MVC5 Controller with views, using Entity Framework’. Now, choose a model (for which you want CRUD), then choose the database context and finally give it a controller name of your choice. After this, sit back and watch the magic happen.

Object Relational Mapping ORM

In an earlier post, I have written a little bit about the relationship that exists between databases, data models and the middle guy in between (or middle girl if you prefer) who may connect these two. It is that middle girl (or middle guy) I am talking about here.

These middle agents are called Object Relational Mappers or Object Relational Mapping systems or some other, much more inventive name that you could come up with as you read this blog post. There are two words here that will help us understand them better.

·         Object – This refers to the objects (instances of the data model classes that we have defined in our application) that we are using to store data in our application.

·         Relational - This refers to the Relational Models that is used in all databases that we use these days.

As the title indicates, Object Relational Mapping or ORM is all about mapping the stuff in the objects (in the application code) into that of the database schema.

I am no expert on ORM, but I understand enough to get by at my work. ORM allows the application developer to work with almost no knowledge about the workings of the database. For instance, it is possible for an application developer to build an entire web application using asp dot net and EF without ever having to write a single line of SQL query.

That is because, the developer (such as you and I) will write code in c sharp, which is what we are good it. When it comes to database related stuff like CRUD (read about CRUD here), you have systems like LINQ (which allows you to write queries to your database context) and EF (which acts like the middle agent and connects your application to the actual database). Thanks to these two systems – LINQ and EF – not a single line of SQL needs to be written.

All this sounds great for folks who have nightmares about SQL (and those beautiful, glorious and sometimes insanely complicated SQL queries) itself. By eliminating the requirement to work with SQL directly, a lot of stuff becomes easy.

Further, when the data model (inevitably) changes during the application development and its life, the corresponding database schema also needs to be changed. When using an ORM, it is possible to simply ask the ORM to update the database to reflect the changes in the data model. EF comes with a system of migration. Through migration, it is possible to push the changes to the database, as they are done to the data model.

The same EF also provides with a Seed functionality which is useful if you are looking to push data default data (Data that must already be there for essential functionality) to the database.

Overall, ORM and specifically EF has a lot of benefits. Obviously, as with anything in life, there must be some issues as well. One issue that I have found is that we are all at the mercy of the ORM system when it comes to mapping objects to schema. If there are somethings that the EF cannot do, then we simply cannot do it, and we may have to resort to talking directly to the database. Further, if the EF system should ever be abandoned by the developer, we will be done for. This problem won’t happen when you are directly talking to the database, because SQL queries today will still work tomorrow. However, EF today when dropped may stop working entirely.

Overall, personally, I prefer to work with ORM and EF. They make my life easy and I am going to trust Microsoft (the provider of EF) not to kill EF anytime soon.

What is this CRUD

I have written a whole lot of articles of asp dot net MVC earlier. You can check them out here. Obviously, when you are using a technology such as asp dot net MVC, you are thinking of building some sort of a web application. An application implies that some data is being served by the server, some data is being sent back to the server from the user client (which is the browser) and stuff is being processed. That means, there is a database in play, and that database is being manipulated by the application directly (where in your app talks directly to the database in question) or indirectly (wherein your app talks to middle person such as entity framework or EF and the middle person then interacts with the database).

So, a database is in play. The moment you think of database you think of CRUD, which stands for Create, Read, Update and Delete.

The titles alone should be explanation enough to indicate what they mean to a database. However, a little explanation for those who love extra detail.

For the sake of completion, and examples stuff, I will imagine that we have a web application, and that has a database. The application is for a water bottle factory, and the database is for keeping track of things raw material stock, water bottle stock and employee related stuff and so on.

C for Create

Create usually means, creating a new row in the database. Of course, create could also mean, create a new table and creating a new schema and so on. However, during the normal operation of the application, you are most likely to be creating new rows (in an existing table more like) than anything else.

So, when you do a C operation, you are creating rows in a table.

For example, I have a table that stores the different employees who work at the factory. Suppose a new employee has joined the company. I will probably have to create a new row in the employee table.

R for Read

Once tables have been created, you would probably wish to read the data that is stored in these tables. That is Read and R. It must be included here that Read could mean a lot of things. You could be reading an entire table worth of data. You could be reading a particular row (based on some value in a column) from a table. You could be reading multiple rows spread across multiple columns.

So read is anytime you read stuff from the database.

For example, say the manager wants to know about all the employees in the company. I will ‘read’ all the rows from the table and provide that information to the manager.

U for Update

Once you have created sufficient amount of data in the database, you will notice that you will always have reasons to modify existing data in the database. Although modify is a good word to use when dealing with a database, it can be misleading because deleting a row (or an entire table) can also be considered as modifying.

Update strictly applies to scenarios where some part of the database being modified, while keeping the other parts intact.

For example, one of the employees comes and tells me that when he joined, he gave the wrong date of birth. Then, she provides me with the correct date of birth. I will then go ahead and update it in the row that belongs to that employee.

D for Delete

Ah, the last part of CRUD. As the name implies, deleting rows is what D stands. Just like create, it is possible to delete entire tables, and even entire databases. However, the D here usually refers to the activity of deleting rows in a table. This could mean deleting a single row or multiple rows.

For example, an employee comes to me and says that she is leaving the company. In that case, I will probably have to delete her row from the table. Of course, in a real world, I will simply move her row from this table to some other table called ex-employees. Of course, that would involve making a copy of the row, creating a new row in another table, and moving all this data. Finally, deleting the existing row in the table.

So, there you have it. CRUD for you!

Data Models and Database and Entity Framework

In the blog posts about MVC, I have talked about Models (which are actually data models and you can read about them here). I have also talked about databases and I have written about entity framework (which you can read right here).

Now, it seems like I have been talking about three different things. Which also means, it is easy to get lost between all these three concepts. They are all pretty different and yet, there are all connected. It’s kind of like the move matrix which seems to have something to say about everything from singing to cooking to living to working.

First up, let’s talk about database. Database is the physical server where you actually store your data. Databases are usually located on servers (that means, when you create a database on your own laptop or PC, you are essentially turning it into a sever). There are many brands of servers, and most of them are similar with their basic functionality. You have Microsoft Server (my personal favorite, of course), Oracle, IBM, MySQL, Postgre and so on and so forth. There are way too many brands of servers today and everybody has their favorites and dislikes.

Whenever an application has to work with a database, it will use what is called as a connection string. Then, using that connection string, your application will connect to it, and then you do your CRUD operations (read more about CRUD here) and get your work done. That is pretty much what happens with a database.

Let’s talk about the data models. Data models is your application’s representation of the data it will be storing. This is extremely useful and essential when you are building your application. Data Models are nothing but classes whom’ s objects you will end up using to store data. You will use the full force of your years (and centuries if you believe in reincarnation) of training and superior thinking to design overtly complicated classes to store data. These are the data models.

Now, when it comes to having your data models interact with your database, you can take two routes. You can convert the data in the data model objects (which are, just to reaffirm, objects of your data model classes) into string, and then write them to the database yourself. This is one way of doing it. You do the reverse of that when you get data from the database and pour it into your database models.

Alternatively, you could use a ORM (Object Relational Model) system (about which you can read more about here) to help you do the two-way conversion of data model to database stuff. One such system would be the Entity Framework that I have already spent some time writing about, and you can find out about EF right here. Essentially ORM (in our case, the EF) is the middle guy who sits between your data model and database.

Hopefully, this post gave you an idea about who the three things – Data Models, Database and EF interact with each other.