Visit my website at

Saturday, March 26, 2016

Sitefinity Thunder: installation instructions (and common mistakes)

When you create a Sitefinity website, for example using the Sitefinity Project Manager, there is no need to open that project as a “Website” solution in Visual Studio. This is because the created Sitefinity website already includes a Visual Studio Web Project (CSPROJ) file.

To work with Thunder, you need to download the VSIX Visual Studio extension file, and install it to Visual Studio. Do not install Thunder from within Visual Studio.

Now that Thunder extension is installed in Visual Studio, to enable Thunder on your Sitefinity project you need to install it on the project as well. This will install and update the required services that interface with the Thunder plugin.

Do not install the Telerik.Sitefinity.Thunder package manually using the Package Management Console. (Sitefinity has it’s own Nuget package source, instead use the Visual Studio extension. The extension doesn’t require you to create a visual studio  
To enable your Sitefinity project with Thunder, use the Thunder extension in Visual Studio as follows. Open up Visual Studio (you do not need to have your Sitefinity Project open in Visual Studio for this). Go to the Telerik (in the Main Menu), then Sitefinity Thunder and invokve “Install Sitefinity Thunder Module”. This will open a prompt merely asking you for your Sitefinity’s project location. The URL is the base of the site, not the “Sitefinity” subfolder.

You might need to refresh the site in the browser to enforce any necessary recompilation. Now you can create a Sitefinity Thunder connection in Visual Studio.

 Finally, when you are creating a connection in the Sitefinity connection Wizard:
  1. You can name your connect anything.
  2. The URL must be the root to the site, that is, do not include the url to the backend /Sitefinity.
  3. Use a username and password that has administrator access
  4. Click Test Connection. You will see that the membership providers drop down list is now populated.

From my experience, if you do anything wrong. You’re left with a very brief error message: “Connection failed.” Go figure.

Friday, February 5, 2016

Technology cannot replace people, but people's intellectual output might decline in value

Technology affects humanity at all of the individual, societal, and global levels. On average, people, societies, and countries globally are developing their creativity, innovation and the various mental abilities. We as human being are taking on huge challenges and making long strides in science and various fields on intellectual activity with the aide (rather than in spite of) technology. Wether technology makes us better intellectuals or not depends on how technology is used, it might render someone lazy or too dependent on technology.

The wide range of advanced technology and the ubiquitous nature of which significantly changed the way humans think. We no longer have to worry about making complex calculations, because we have programmable computers for that. We have advanced spreadsheet applications with many features. When we delegate the time consuming tasks of computing and calculation to machines, we are free to focus our effort and time on designing solutions to higher-level problems while we explore bigger and better issues in the various fields of human endeavor.

What does it mean for us to be able to think for ourselves? Does mean our ability to multiply huge numbers in our head without using aids? Certainly someone who haven’t been using a calculator for some period of time, would be better at this than someone who has been “relying” on a computer or calculator to do the multiplication. But then, is that really “thinking”? One might simply argue that this task of calculating, which was historically considered something that only humans can do, isn’t even a task of thinking but a merely mechanical task.

In the past, a Calculator was a job title of an employee! Now it’s a small application on a desktop computer. Instead of having an amazing human with a bright mind spend all his or her time at the Ministry of Finance doing mechanical calculations, we have such people doing work that matters: designing markets, creating effective strategies on how and where to invest, providing insights on the dynamics of local and global economy and these tasks require intelligence and a lot of high-level thinking that machines (or calculators, no matter how complex they are) can’t do.

Technology is still advancing, and more work that we thought only people can do, will be doable by machines. To some this means we will all be replaced by machines with nothing to do. I verily disagree: no matter how intelligent artificially intelligent machines are, we will still have a lot of things to do, but things that matter more than that which can be performed by machines.

Having said all that, there is another view point. Some people are made lazy by machines because they do not grow their thinking to higher levels. For example, if all what needs to be done in some accountant’s job is to perform addition, multiplication and other mathematical functions on some numbers and then he or she starts using a computer program that does exactly that, and he or she does not add value beyond what the program does, then not only his or her ability to do math will weaken and wither with time, but so will his or her value.

When something becomes automatic, we need to search to do more and provide more value than what can be provided by technology.

Saturday, July 25, 2015

Optimization is in the details

I learnt the following from one of my former managers whose style I didn't agree with, which reminds me of something that my father used to always tell me: "You can learn from anyone. Learn to avoid the incorrect from those who fail, and learn to do the correct from those who succeed."

My manager used to avoid looking into details, despite having the opportunity to optimize there. The assurance that things are working correctly and efficiently is in the details. To do the job right, and to be able to make correct decisions, one must be informed about the details of how the things they want to make right works. The more detailed relevant information one has, the better the decisions they are able to make, and the more opportunity they have to enhance things.

Avoiding communicating details hindered the team's progress because of arbitrary decisions that are not technically optimal and at times just incorrect. If one doesn't want to look into the details, then they should not make decisions. Not only should they consult those who are familiar with the execution details when they want to make decisions, but they should also delegate the whole decision making process to them at times since they are the best suited for this.

Having said that, this means that decision makers should also know in details the capabilities of their team and know how they work and how their decisions affect their efficiency. And keep a healthy communication channel to facilitate the exchange of information and use that channel eagerly to ensure that relevant information is communicated and done so early on.

How can you make technical decisions if you don't know what an API looks like and how it is used?
How can you fix something if you don't know how it works and be familiar with its components and their interactions?

But details are not a burden that gives someone a headache. Details are beautiful because without them the things you are dealing with would not exist. Details make the things that we see and interact with every day. There are details in everything, and if you have the curiosity and learning about things stimulates you, then details will not be a burden but an interesting, enjoyable, and beautiful thing to deal with.

Of course details must be relevant and not overwhelming, otherwise they might distract you from your primary goals. This is where delegation comes in handy. But delegation does not mean to throw work at your subordinates and go sleep. Managers have to closely follow up with the purpose to assist and not just blame and hold accountable their employees. Anything motivated by laziness or lack of concern backfires.

Tuesday, June 30, 2015

SharePoint List Template with Lookup Fields

If one creates a list template that has a lookup field to another list (the reference list), and then attempts to create a new list from that STP template in another site, the data in the lookup field of the new list will be empty. Of course the obvious reason is that the reference list does not exist in the new site.

The problem is that even if you create the reference list (or import it with another list template) in the new site, that won't simply work because the lookup field defined in the STP file refers to a list through a GUID that is valid only on the site from which the STP template was originally created.

Even though I created a list with the same name, it has a different GUID.

The solution to this is:

0-Import reference list to the new site, and note its new GUID. Open the STP file and replace the GUID of the original reference list with the GUID you just noted, this is how you can do that:

1-STP files are CAB files: so you can rename the template file's extension to be able to open it as a CAB file. You can verify this by openning the STP/CAB file and finding that it is a BINARY file that starts with the letters MSCF (Microsoft CAB Format).

2-Extract the Manifest.xml file from the CAB and open it in an XML editor. This is a big XML file, so I rather open it in an XML editor e.g. Visual Studio. Note that even you open it in an XML editor, it is not pretty formatted by default.

3-Find the definition of the lookup field. It will be something like the following. I have a Multi lookup field, so the lookup field type says LookupMulti. I found this field by searching for the GUID of the original reference list as you can find in the List attribute. I then replaced it with the correct GUID that applies to the site to which I'm importing my list with the lookup field to.

<Field Type="LookupMulti" DisplayName="RequiredTrainings" Description="The trainings required for the corresponding job title." Required="FALSE" EnforceUniqueValues="FALSE" List="{ddfd206e-206a-405b-8926-94447e2c3d09}" ShowField="Title" Mult="TRUE" Sortable="FALSE" UnlimitedLengthInDocumentLibrary="FALSE" RelationshipDeleteBehavior="None" ID="{165f9634-9a76-4c4f-b1f0-9b1e09e9f2ed}" SourceID="{56091bc8-c44c-438c-a114-18e283cd3ca4}" StaticName="RequiredTrainings" Name="RequiredTrainings" ColName="int1" RowOrdinal="0"/>

4-Recreate the CAB file. You can use makecab DOS command.

5-Rename the cab to STP and import it to the site, then create a new list from it!

Tuesday, June 9, 2015

ID Query String parameter in SharePoint Visual Web Part in a Wiki Page

What happens when you access a query string parameter in a visual web part?

A web part is just part of the ASP .NET SharePoint page, so one could safely assume that it is an ordinary and a correct thing to get the value passed to the page through a query string parameter. The visual web part control has access to the Request object which contains the data passed to the server as part of the request, including query string and form data. Of course the web part control and its child controls rely on the Web Forms View State, and that is also passed with the request.

It took me hours of experimentation and guesswork to realize that creating a visual web part that expects a parameter that is innocently called “ID” makes something under the hood, go haywire. I haven’t reversed engineered SharePoint to realize this, but it’s surely not just an ASP .NET issue. Because of course ASP .NET has no reason to restrict you from using any query string regardless of name.

SharePoint on the other hand, seems to treat a request with an ID in the query string, in a special way. ID is the same query string parameter used to show List Items, such as the pages to Edit and View list items. Request to these pages looks like: http://mysharepoint/sites/mysite/mylist/view.aspx?ID=1. And what I was trying to do was just let one web part page create links to another page with ID passed in the query string, and that page would have another web part that checks for the ID passed.

The problem happens specifically when you request a Wiki page with the ID parameter. In my scenario, I created a couple of web parts and added them to different Wiki Pages. Then they link to each other using the ID parameter. The issue happens even if you don’t use web parts at all. Just create a Wiki page, and add ID=1 to its query string (e.g. http://mysharepoint/SitePages/TestWikiPage.aspx?ID=1).

It simply doesn’t work and one must choose another query string parameter name. Choose for example: MyID or CustomerID instead of ID. Read about another SharePoint mystery here: SharePoint application page 404 error!

Try it! Just create request a SharePoint Wiki page with the ID parameter passed as a query string parameter in the request URL. Please let me know if it magically works for you, because for me, it doesn’t work even if I don’t use web parts or any kind of customization. This is one more example of how the various SharePoint components interact in complex subtle ways creating such unexpected behavior. Things should be simpler and I talk about that here: Diagnosing SharePoint Errors.

Tuesday, June 2, 2015

SharePoint Application Page results in HTTP 404 Error!

This is an example of how perplexing the interaction of the different SharePoint components creates confusing debugging scenarios for developers. I describe the process I went through to diagnose an elusive HTTP 404 error. If you just want the solution, find the section titled “The Solution” near the end.

I had a working SharePoint farm solution with two application pages. I developed the relatively simple solution mostly using Entity Framework. I understand that touching the SharePoint database is an anti-pattern, anything that goes around the official reliable API is a bad anti-pattern. I wasn’t touching the database, just looking J. It was an experimental project anyways. But which database I was accessing, is not related to the issue I was facing:

I deployed the application successfully to a different server. Everything is working fine.

The environment of the test server changes much dynamically than a relatively static very stable production server. For some reason, going back to this project after a while, I run the solution and navigate to one of the SharePoint application pages. To my unpleasant surprise, I get an HTTP 404. Everything is working except my two pages.

What could be going wrong? The solution is deployed successfully and I can find my application pages in the SharePoint layouts folder (defaults to C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\TEMPLATE\LAYOUTS\). I traverse the possible-solution-tree or graph, no, it’s actually a maze (at least to someone like me who is not a Microsoft MVP SharePoint consultant with 15 years of experience).

I added try-catches everywhere, even though I had a big try-catch wrapping the main part of my code. No exception was caught, so I thought there is some exception handled by an internal component of SharePoint, and that component is totally invisible to me. Speaking of S = A + B in one of my previous posts, we have S = A + B + C where C is a component that is just invisible to you!

I also tried to recreate the application pages and re-adding them as maybe some SharePoint project build configuration got messed up somehow, then I tried to recreate the whole project to the same result: HTTP 404. I was stunned because there is no configuration or deployment issues and no apparent problems with my code, which was working before!

Suspecting again the problem is with the code, I created an empty SharePoint application page to my project, and to my surprise it worked. This points that I am indeed doing something wrong with my application pages, yet no exception is thrown that I can catch! And what is this HTTP 404 for an application page that is actually there?! Chrome’s debugging console simply just shows that the application page is being requested but not found.

I remember how sometimes Visual Studio debugger fails to catch some kind of errors, either because it is configured not to do so, or it just fails: COM Exception and Visual Studio crashing. But this doesn’t seem to be a Visual Studio problem.

Then I checked the SharePoint logs… What a nightmare trying to find your way there.

What I’m trying to illustrate here is that checking the various components in a system in an attempt to diagnose errrrrors involves considering different scenarios and options. I tried many options until…

The Solution

I discovered that the primary difference between my application pages that caused the server to return a 404, and the other empty application page is the reference to EntityFramework. When the runtime tries to load EntityFramework.dll, a System.IO.FileNotFoundException is thrown because the DLL is not deployed in the right way with the solution.

A SharePoint solution is dissimilar to a regular Web Application project in the way it is deployed. When a web application is deployed, the application can find the libraries that it references in the same location of the application’s DLL or in the machine’s GAC.

In my case, and for some reason, the SharePoint solution was not able to find EntityFramework.dll. It’s a file, and because of the System.IO.FileNotFoundException, either ASP.NET or SharePoint environment found it appropriate to just return a 404. This is simply does not follow the HTTP protocol where a 404 means that the client requested the wrong location. The server not being able to find files required for the page to operate correctly is in fact a 500 server side error!

Deploying EntityFramework.dll in the GAC solved the issue. I must remark that part of the possible-solution-tree has something to do with running Visual Studio with Administrator privileges.

Monday, June 1, 2015

SharePoint: Thoughts on Diagnosing Issues in a Complicated ASP .NET Web Forms Application

SharePoint is an ASP .NET application, built with many out of the box enterprise collaboration features and a sophisticated customizability model (SharePoint solutions). Unfortunately, it’s an ASP .NET Web Forms application.

Microsoft’s ASP .NET team realized the importance of a clearer and cleaner web development model wherein control is given back to the web developer with ASP .NET MVC. It is a great alternative to ASP .NET Web Forms. Many ASP .NET developers struggled with Web Forms to create web solutions that required more granular control over the outputted HTML and JavaScript (of course there is the advantage of black-boxed web controls and third party products that make a lot of things easy in Web Forms: that’s another thing to consider). One can find lots of thoughts comparing the two on the web.

Although many find ASP .NET Web Forms limiting in many ways, it’s the technology used to build SharePoint as SharePoint predates ASP .NET MVC. Building SharePoint applications using ASP .NET MVC is a different story; you can customize SharePoint by building SharePoint Solutions that use ASP .NET MVC. What I’m talking about here is the system itself: the built-in features and tools. For example, the display, edit, and new forms for lists and items and the master pages in SharePoint are ASPX pages, Web Form pages. In addition, Visual Studio Tools for Office, which includes SDK libraries for SharePoint and project templates for SharePoint solutions, has Web Forms ASPX pages with code behind, and a Visual Web Part project references System.Web by default (System.Web.dll contains the System.Web.Forms namespace).

I think that using Web Forms and the accompanying limitation in a clean architecture is one important factor that makes SharePoint such a hard thing to master. I imagine a much better SharePoint had it been developed using ASP .NET MVC. Technology choices affect architecture which surely affect developer experience in turn. Componentization is part of the process of creating a good architecture, and SharePoint is full of components running here and there (including Windows services), and componentization is only one part of a good architecture and it does not necessarily mean clarity. It also certainly does not mean clarity for the developers who are implementing and building on top of SharePoint.

Diagnosing failure is one primary task of IT specialists and developers. Figuring out why something occurred or behaved the way it did is one vital part of such diagnosis, and wandering a complicated world of componentization and the resulting exponentially complicated possible-solution tree is not a trivial task. SharePoint is not a trivial system and so is diagnosing its issues. I believe I’m not the only one who struggles all the time trying to find out the root cause of a problem.

Components interact as they function. When something fails, the obvious is that the problem could be in one or more of these components. But beyond figuring out whether each of the components is working or not, the interaction modes of those components within contexts (static or dynamic) is also something to consider. For example: suppose a system S composed of components A and B fails. A is working fine, and so might B. But beyond consider that S = A + B, there is an environment E, the host of relevant configuration and other elements, within which A and B might not be able to operate together correctly. This means that the components are fine, but the configuration of the environment causes only their interaction with each other to be faulty. We might also need to check whether the environment has hidden interfering elements, X, Y, and Z that has components in related environments.

As I said before, the search space is huge with SharePoint.  I went into the abstract here as this applies to any system, but I’m trying to illustrate the point of why SharePoint is tricky to master, and why is it hard to diagnose it and see through it with clarity and confidence, at least for the non-masters who have been SharePoint-ing since more than a decade.

The thoughts here are provoked because of what happened to me a few days ago while debugging a SharePoint application page.

I will blogged more about that in another post here (404 error on Sharepoint page with ID parameter), but what I faced was an interesting issue with one of my custom SharePoint application pages, wherein I was getting a Not Found (HTTP 404) error in the browser when requesting an application page that surely existed. At first I thought it was some deployment issue as perhaps one of the test deployments remained in an inconsistent state for some failure yet to be diagnosed. Or maybe my code was causing some underlying API to throw an exception not accounted for which was not catchable for some reason by a try-catch. I went on searching for a solution, traversing SharePoint’s complicated possible-solution-tree. The reason that I found surprised me. I will write more about that in a subsequent post.