Showing posts with label Projects. Show all posts
Showing posts with label Projects. Show all posts

Sunday, October 30, 2011

JOCU – The Alpha

I’ve been working on an OData reader in java for some time now, and I am ready to show people the first alpha.  There are some pieces missing, and I would prefer better ways of accessing some of the data.  But here it is.

The release only has the underived version of the JOCU project.  It allows for most of the functionality from the OData protocol (the main one missing is the data format, this uses XML data src as apposed to the json format).

Here is a code snipit I wrote that uses the API from the JAR library.

import java.sql.Timestamp;
import java.util.Date;
import java.util.List;

import lkoutzas.jocu.shared.AbstractCatelogueData;
import lkoutzas.jocu.shared.AbstractEntity;
import lkoutzas.jocu.underived.ODataEntity;
import lkoutzas.jocu.underived.ODataLink;
import lkoutzas.jocu.underived.ODataRepository;


public class RunJOCUDemo {

public static void main(String[] args) {

System.out.println("Welcome to the demo of the JOCU odata reader!!");
System.out.print("Catelogues Start: ");
System.out.println(new Timestamp(System.currentTimeMillis()));

// create the odata repository
ODataRepository oData = new ODataRepository();
// set the URI for the location of the repository
oData.SetURI("http://services.odata.org/OData/OData.svc/");
// get oData Catelogue
List<AbstractEntity> categlogues = oData.FetchODataCatelogues();
System.out.print("End: ");
System.out.println(new Timestamp(System.currentTimeMillis()));
// dump out results to show it works...
System.out.println("Catelogues...");
for(int i=0;i<categlogues.size();i++)
{
AbstractEntity local = categlogues.get(i);
System.out.println(local.getAttribute("href"));
}
System.out.println();
System.out.print("Entities Start: ");
System.out.println(new Timestamp(System.currentTimeMillis()));
// create the odata repository
ODataRepository oData2 = new ODataRepository();
// set the URI for the location of the repository
oData2.SetURI("http://services.odata.org/OData/OData.svc/");
// set the catelogue for the oData repositpory
oData2.SetCatelogue("Categories");
// query or filter the data request (need to detail full list)
oData2.getQuery().top("2").select("ID,Name");
// get entities
AbstractCatelogueData entities = oData2.FetchEntities();
System.out.print("End: ");
System.out.println(new Timestamp(System.currentTimeMillis()));

System.out.println("entities...");
for(int i=0;i<entities.getEntities().size();i++)
{
ODataEntity local = (ODataEntity) entities.getEntities().get(i);
System.out.println(local.field("ID") + ": " + local.field("Name") + " links are: ");
for(int j=0;j< local.getLinks().size();j++)
{
ODataLink odataLink = local.getLinks().get(j);
System.out.println(odataLink.getTitle() + " - " + odataLink.getHref() + " ("+ odataLink.getRelationship()+")");
}
}
}

}


As you can see, the API is pretty easy to use and I think pretty good.  So, for the “bits not in the alpha”.


Firstly, there is a bug that will mean that XML structured elements (CDATA as an example) are not parsed correctly.  They get broken down into separate elements rather than a single blob.


Also, I would like to improve the manner that the data is being accessed.  It feels a bit clunky to me. I am not sure how I can improve it, but I would like to see what I can think of.


The last missing bit / bug is that the expanded entities within the querystring.  At the moment I think it would fail badly.  The API will allow it, but the data will likely either cause the call to fail or it will will be ignored.


Once these items are in the code, I am going to work on the derived version, which will allow strongly typed objects and (hopefully) reduce the size of the downloaded packet.


The link for the alpha is here.  Let me know your thoughts.

Sunday, March 6, 2011

JOCU – The Final Stretch

So, I have the basics of an XML parser completed, now I need to implement this to deliver what I am wanting, which is the Java OData Consuming Utility.  I am going to work on the loosely typed or underived version first, then I will implement the derived version.

Briefly, lets look at the functionality we can expect from the repository. 

string GetSource() / SetSource(string)
This will define what the source of the oData is.

bool GetMetadataRequired() / void SetMetadataRequired(bool)
This will set or get if the metadata is required for this repository

ODataEntities Fetch()
This will get the entities from the oData query

IODataQuery Filter(string)
Applies the filter URI convention to the source

IODataQuery SetEntityCollection(string)
This will define which collection that it is desired to get

IODataQuery GetEntity(string)
Gets a single entity from the collection

IODataQuery SetOrderBy(string)
Sets what the list is ordered by

IODataQuery Top(string)
Sets the number of entities that we want returned

IODataQuery Expand(string)
Sets which related entities to expand

IODataQuery Format(string)
Sets the format to return the data in (not used really for what I want, should I scrap it?)

IODataQuery Select(string)
Identifies which entity items to select, and related items

IODataQuery InlineCount(string)
Sets the inline count property on the data source

IODataQuery CustomURIElement(string)
Sets a custom element to the query that is sent to the OData source

string GetMetadata(string)
This will get the OData collection metadata entry

And, from the point of view of the entity we have the following:

string Get(string)
Gets the value of the key passed though

string GetMetadata(string)
Gets the metadata value for the key passed though

Most of this is derived from the OData URI conventions, with some basic access functions thrown in.  I think that this is all I need to implement, but to be honest I’m not sure.  I have a feeling I will be uncovering more than I expect as I start to tackle the implementation of the repository access, but I need to start somewhere, and this is the best place.  I am aiming to finish this off in the next month or so, but as I am doing this in my spare time to have some fun we shall see what happens.

Saturday, January 22, 2011

Agile vs Waterfall

Recently I found myself coding less and “planning” more. It took me a while to work out why, and when I did I thought I would write this so that others may avoid the trap I was in.

Firstly, some background. As you may or may not be aware, I am writing an API in java for read an OData source.  Now, I am using TDD principals to code the API (basically write a test case and then write the code that makes this test pass).  This means that I need to think in small increments, write a test for the small increment, then write a small increment that satisfies that test, then write another test for another small increment, then write that small increment and so on,  you get the idea.

Now back to the present.  I encountered the next part of my code requirements, implementing XML namespaces within my code.  This is where my problems started.  I did the right thing by researching some things on XML namespaces, to make sure that I understood how they were implemented and make sure my code didn’t vastly differ what it should be.  Then I did the wrong thing, I tried to plan the entire implementation of the XML Namespace component, rather than looking at the small piece I was dealing with (in this case pulling the namespace and storing it in the required format).

I was planning the reading dtd and metafiles and how that would work.  Looking at the manner that dtd’s could be delivered.  How would I know which was an internal namespace and which was an external? How would I register the external namespace in my code (for example the $metadata of an OData source)? And so on and so forth.  That is not how I am wanting to write my code.  I want to write a test for a small part, then write that small part.  In this case the small part was identifying the namespaces and putting them into a useful format.

This got me thinking about agile and waterfall practices.  Agile methodologies would say that you get the requirement and break it down into smaller pieces, and those into smaller pieces until you have something that can be done in a defined period of time (for example, in scrum you would break it down into smaller pieces that would be done within an iteration).  Whilst you know what the requirements are, you don’t design the entire solution for what may or may not happen, nor do you plan everything out at the start.  You break the requirement into smaller chunks and that into smaller chunks into a requirement that you can complete within a week or 2.

Waterfall is very different to this.  In the waterfall approach, you define every requirement that you can think of, plan the end result to the finest detail, then you start coding to the fine details that you have planned to deliver.  You don't start coding until you have the UX defined, the web services documented and the classes worked out and what each class properties and methods are.

Now, without going into a discussion about waterfall vs agile development practices, I will say my thoughts.  While there are parts of one in the other and vice versa, the two can not coexist.  For example, when your doing agile you need to know the end result, what is the stakeholder asking for.  That you can break down into smaller requirements and so on until it is small enough to complete in iteration or sprint.  And conversely in waterfall development, you can not say “deliver the whole project at once”, you need to break it down into smaller chunks that a developer can work on.

The main point of difference for me is that in my opinion, agile is about meeting the requirements that you have been asked to deliver.  No more no less.  You cant tell the stake holder what they want, you can only guide them along as you code and as they see the work.  This doesn't mean that if they say they want a field you don't do any error checking on it, or that you just make it work and not deliver a quality product.  What it does mean is that you don't enlarge the requirements to what you can see being needed down the track.  You meet the current need. If the database table needs a title, description and timestamp, don't add a link to a status that is then part of a different requirement.  By the same token you can guide the requirements along by asking questions like “where is this going to be reported?”, “Is there a need to report on this?”, “ Who should be able to edit this?”

Waterfall projects on the other hand you can vastly change the requirements because you are part of the planning process.  Nothing gets done until everything is defined and some form of specification documented.  You research to make sure that it is future proof, so that you can work on another project and this one wont need any attention.  You tend to guide the requirements and specifications, rather than the development.  But, once it is locked in, it takes a lot to change one line of the specifications that your working to.

Personally, I prefer agile development processes over waterfall process because you can be flexible, if you code something that the user changes their mind on you can adapt. (Hence, why it is called agile.)  You also only deliver what is needed, not what you think is best.  The issue with stakeholders is that they don’t always communicate it well to developers, project managers or product owners.  Hence the need to ask questions and listen to the answers.  You also end up with leaner code (it does what is needed, and nothing it doesn't) rather than bloated code with parts that are rarely, if ever, used.

Saturday, May 22, 2010

The First One

So, I have re-named the data structures.  I used the database project template in Visual Studio 2010, and to be honest I wasn’t impressed. As a database management project tool, I found it almost impossible to commit the updated changes back to the database.

But first, let me look at what I did and what I liked and what I didn’t like at each step.  My first step was to use a clean install of the project and get that imported into the database project.  That worked pretty well.  It created one file for each database object (stored procedure, table etc.) which I thought was great.  It created a good structure which was pretty easy to understand (for a non DBA like me).

VS2010 SQL Project FolderStructure

To me, this structure is very similar to the structure that you see in the MS SQL Studio, which is a good thing in my mind.

Then I did one of the items on my version 2 list for the project, to rename the database tables.  The refactoring was very much in line with the way code gets refactored in VS2010.  It was quick and easy, and every reference was renamed.  Needless to say I did like that.

Now to the parts I didn't like.  I wanted to set one of the tables to save the data within as it was part of the project, not the dynamic data of the system usage. I could not work out how to do this.  I would have thought that a database project would have been able to allocate some default data to the tables, if not mark some tables as having all their data from the project loaded into the implementation of the project.

This leads to my next problem.  I got the source data structure from a database.  I made a relatively simple if somewhat extensive change.  I then wanted to update the database with the changes and it would not update.  Kept on returning a user error or something.  I am going to be honest here.  This to me is the core of a database project.  Writing scripts is one thing.  The ability to use a GUI to manage the code is awesome.  The UX for updating the data structures is not.

As a result, at this time I am not going to use the database project for my changes.  At least until I am able to resolve the 2 issues that I have encountered so far.  So, at this stage I have renamed the database tables, and am about to tackle another of the items on the list.

Saturday, May 15, 2010

A Breath Before the Start

Well, I had a friend look at my blog the other day. He is someone who I respect in the technical world, knows heaps, and is just an all round good guy.  (For those wondering, the man is Richard Banks, who you can find here.  He is also worth a follow on twitter, over here.) Anyway, he pointed out some spelling and grammar issues, which I was expecting.  He also gave me some tips and tricks for blogging, like the code colouring and where I can host files without any ads, which I will incorporate I to my last few posts.  I'm not going to tidy every post, just the last two. Hope to do that this weekend.

Then he ended his advice with a “but”. He wanted to know why I was looking at using prefixes on table names, and my thoughts on a "decoupled" application. And he recommended that I review some other patterns, such as the repository pattern. Taking this advice, I did some searching on the web to look into the repository pattern, and as he suggested I reviewed my thoughts and the post content.  Following this, I have decided that he is right (as he usually is).

As a result, I am going to make some changes to my approach.  Firstly, I am going to write an updated version of my audit code.  One without named prefixes, that makes more sense with regards to naming conventions.  (As you can see I am dropping the prefix naming convention on the data layer.)

Secondly, I am going to write a decoupled application in the correctly.  One that has it’s layers decoupled, not the modules within each layer.  This means some changes to my intentions, but I can make that work.  I am still going to maintain the audit functionality in a separate database from the Line of Business database, and I will explain my reasoning's for that when I release a new version of that little project.

Thirdly, I am going to write code that has a designated purpose.  I am going to achieve something that I can use as a vehicle to learn.  It has been said that without vision people perish.  Probably from not having a goal and aimlessly wandering around til they did. I am hoping that with a goal to aim for, I can be more focused on doing and learning rather than wandering down a dark alley.

As an aside, this is the point of me coding and writing blogs about what I am doing and my thoughts on it all.  I am wanting to learn, and I just happen to be in a position where I can ask people to check my blog and code.  If one stops being teachable, stops wanting to learn, then they are, in my opinion, not worth listening to.  I don’t want to be one of those people.  Other than fixing formatting issues and typos, I wont be altering or removing the previous posts.  It is a reminder to me of mistakes I have made to teach me not to make them again.

So, my next post will detail what I am aiming to do with my audit scripts, which I will do try to get out as soon as I can.

Thursday, July 16, 2009

Just Keep at It

When projects start to look like they are going to be late, not be delivered or seemingly impossible to complete, there is one phrase that you need to hear.

"Just keep swimming, just keep swimming"

Yes, this may come from a children’s movie, but it doesn’t make it any less important. When all seams lost, don’t jump ship, just keep going. Focus on what you need to do next. Let people know about the issues that are causing the delivery date to slide, or functionality to not be delivered. As the saying goes, the only way to eat an elephant is one bite at a time.

And trust me, things are likely to get worse before they get better. It is the nature of things. Call it Murphy’s Law, it doesn’t matter. The main thing is that you need to keep at it. You can get though; just try your best to stay positive. Get positive people around you who can encourage you to get though.

Anyways, I hope that to those who are in the middle of one of those projects this lifts your spirits and your eyes can see the light (even if it is the size of a pin hole) at the end of the tunnel. And no, it isn’t an oncoming train. :D

Thursday, July 2, 2009

It's all in the detail

Have you ever wondered why you complete a project and not gotten the pat on the back for all the hard work? When that happens to me I generally find it is because I have provided the overall concept of what was wanted, but the fine detail was missing.

By detail I mean the simple things like getting the User Experience right. This includes wording and correct spelling (yes, spelling is important); but also the pecieved way that features should work as apposed to the actual way that they work. It might do the same thing, but users generally know what they expect to see, even if they can't communicate that with you.

So, to finish, a word to the wise. While you need to make sure you deliver the project or functionallity, make sure you don't forget the details that will really make the UX just that bit better.

Monday, June 15, 2009

Understanding

Understanding what you need to do before you start a project is important, but not understanding it all shouldn't stop a project starting. Most project work starts with a phrase like "wouldn't it be cool if we could....." or “I would like to be able to ... so that I can ..." I'll let you fill in the blanks.

This start leads into a review of what exactly is required, how it might work, and what the end user can expect when the project is complete. The more complete the review and requirements list is, the better for the understanding of the people working on the project. However, you can start the project even without this review being complete. I would say that it needs to be 60-70% complete, but at that point the project work can start.

Here is the catch; to start a project where the requirements are not complete invites a risk that you might need to redo parts as the requirements get more and more refined. If you’re willing to take that risk, then there is no problem. Just be prepared to make changes as the requirements change, or when new ones are added.

Why start when you might just need to redo it all? If it is done well, and you are able to mitigate the risk, you can deliver the completed project earlier than if you waited for the entire review to be completed. The risk could mean that it takes longer as you are always redoing the work to fit the latest requirements.

Tuesday, June 9, 2009

Direction

Direction is very important. It is one half of velocity, is a part of vision, and can help you work out where you are. In a project situation, the direction the project takes starts at the very start of the project.

What happens when you don't have direction? There is no purpose, so why continue. There is no confidance that your doing the right thing, so why bother. Without direction you don't know where your headed. You don't know if your using the best tools to get the job done right, first time.

With direction, on the other hand, you can make decisions that are going to get you in the general vicinity of where you want to be. You can plan what needs to be done before you do it, not react to changing tides of what people want.

However, there is one important thing about direction. (Well, probably more than one, but this one is up there.) You can't just start moving in any direction. You need to know where you are headed, atleast generally speaking, so that the direction you take helps you get to where you want to go. Otherwise the gain you have in moving becomes a liability, and can get you into as much trouble as doing nothing at all.

Direction by itself is not good. It needs to be coupled with vision, or what the end goal is. This can be something simple, like a project and a deadline, or more complex like a 5 year plan. But without direction you will just be chasing your tail.