GAE - Datastore filtering using IN operator

Since I can't find in datastore queries the documentation on how to use the IN operator, that why I decide to share my expirimentaion.

Here how I did.

Query q = new Query("YourKindHere");
// Create a list of possible data tobe compare in your IN operator
List<Long> levels =  new ArrayList<Long>(); // Initiate a List object
// Add the possible levels
// INFO: If you notice, I put "l" to represent the value as Long
arrayList.add(1l); // add level 1
arrayList.add(2l); // add level 2
arrayList.add(3l); // add level 3

// Create and AND operation with IN operators
q.setFilter(CompositeFilterOperator.and(
  FilterOperator.EQUAL.of("active", 1l), // compare as active=1
  FilterOperator.IN.of("level", levels) // compare ad level IN (1, 2, 3)
));

For other info about queries please check google appengine datastore query docs. Java DataStore Queries.

GAE - PreparedQuery.TooManyResultsException

Just take note before using asSingleEntity you must to make sure that you query will on return one entity(record) or else you will came up a problem TooManyResultsException

Suggestion to limit this unexpected behavior

public static Entity getSingleData(Query q){
  FetchOptions fOption = FetchOptions.Builder.withLimit(1);
  try{
   return datastore.prepare(q).asList(fOption).get(0);
  }catch (IndexOutOfBoundsException e) {
   logger.warning("No data sorry: "+e.getMessage());
   return null;
   
  }
  //return datastore.prepare(q).asSingleEntity();
 }

GAE - How to to handle DeadlineExceededException

Before we do. Lets recap how this Exception encounter or what is the cause?

This Exception encounter when a request runtime exceed the 60 sec runtime. Some you are working with loop.

Lets take some example. Somehow you are looping some data and update some properties of every record.

Possible solutions/workaround

Somehow you must have extract servlet the equivalent process to every record. Lets take you have servlet /updateData and /updateDataBackend. You updateDataBackend will process the other data that will possible reach by Deadline. By the use getRemainingMillis() method you can able determine if the extra will be reach by Deadline.

/updateDataBackend

import com.google.apphosting.api.ApiProxy;

...

for(Entity ent : myDataEntities){
  if (ApiProxy.getCurrentEnvironment().getRemainingMillis() > 5000){
     // I think 5 seconds can still do the job
  // Update data
  }else{
    // Opps, It almost reach DeadlineExceededException better create a backend procress
    // for extra data
    //  Please the updateDataBackend to run in backend
  }
}

Hope that helps. Also try to read The_Request_Timer

Happy reading

Getting Stated on Spring MVC with Google AppEngine

Since, Appengine was the best free java server to deploy application. This is why good to integrate java frameworks to google appengine.

As this point, assume that you already create your Google Appengine project. However if you don't know how to create GAE project go here.

Pre-requisites:

* Eclipse Kepler + Spring 3.2.4 + Google App Engine 1.9.7 + Java 7
* Download Spring jars

Extract the zip files and copy jars to war/WEB-INF/lib/(for deployment) and include the jars in project library.

1. Create mv-dispacher-servlet.xml.

<beans xmlns="http://www.springframework.org/schema/beans"
 xmlns:context="http://www.springframework.org/schema/context"
 xmlns:mvc="http://www.springframework.org/schema/mvc" 
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="
        http://www.springframework.org/schema/beans     
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/context 
        http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/mvc
        http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd">
 
 <!-- 
        Component 
        https://developers.google.com/appengine/articles/spring_optimization
 -->
 <!-- 
 <context:component-scan base-package="com.winzter.springmvc.controllers">
  <context:exclude-filter type="regex" expression="com.winzter.springmvc.controllers.GuessController.*" />
 </context:component-scan>
    -->
 
 <!-- DEFINE Your bean Here -->
  <bean class="com.winzter.springmvc.controllers.GuessController" />
  <bean  class="org.springframework.web.servlet.mvc.support.ControllerClassNameHandlerMapping" />
   <bean class="com.winzter.airgames.bigbarnword.controllers.FriendsController" />
 
 <mvc:annotation-driven />
  
 <!-- Bean to show you Di in GAE, via Spring, also init the MovieController -->
 <bean class="com.winzter.springmvc.controllers.GuessController">
  <property name="message">
   <value>Hello World</value>
  </property>
  <!--constructor-arg name="message" value="Help ET go home" /-->
 </bean>
 <!-- bean class="com.winzter.airgames.bigbarnword.controllers.FriendsController">
  <property name="message">
   <value>Hello World</value>
  </property>
 </bean-->
  
 <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver">
 
 <!-- 
  This serve for JSPs
  Every controller menthod return string, it file /pages/string.jsp 
 -->
  <property name="prefix">
   <value>/</value>
  </property>
  
  <property name="suffix">
   <value>.jsp</value>
  </property>
 </bean>

</beans>

2. Update/add code below to web.xml

<servlet>
  <servlet-name>mvc-dispatcher</servlet-name>
  <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
  <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
  <servlet-name>mvc-dispatcher</servlet-name>
  <url-pattern>/</url-pattern>
</servlet-mapping>

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>/WEB-INF/mvc-dispatcher-servlet.xml</param-value>
</context-param>

<listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>

GAE- The method addFilter(String, Query.FilterOperator, Object) from the type Query is deprecated

In this situation, we can fixed this by changing out code.

Import the Libraries

import com.google.appengine.api.datastore.Query;
import com.google.appengine.api.datastore.Query.CompositeFilterOperator;
import com.google.appengine.api.datastore.Query.FilterOperator;

Sample:

Query query = new Query(Kind);
query.addFilter("active", FilterOperator.EQUAL, 1l);
query.addFilter("user", FilterOperator.EQUAL, _user);

Solution1:

Filter activeFilter =
  new FilterPredicate("active",FilterOperator.EQUAL,1l);
Filter userFilter =
  new FilterPredicate("user",FilterOperator.EQUAL,_user);

// Combine the Two filter
Filter queryFilter =
  CompositeFilterOperator.and(activeFilter, userFilter);

// Use class Query to assemble a query
Query q = new Query("Person").setFilter(queryFilter);

Solution 2

query = new Query(Kind);
query.setFilter(CompositeFilterOperator.and(
  FilterOperator.EQUAL.of("active", 1l),
  FilterOperator.EQUAL.of("user", _user)
  )
);

If you want look more info you cant find Datastore Queries and GAE Query

Happy Reading

GAE - How to download your code in appengine server.

Here the simple way to do.

PATH_OF_YOUR_APPENGINE_SDK/bin/appcfg.sh --application=YOUR_APP_ID --email=xxxx@gmail.com download_app war

GAE - Setting mimetype manually in Web Apps

Google Appengine Static Files

In some static files. Google Appengine send a wrong mimetype and mostly set application/octet-stream.

To be fixed the minetype, we need to configure the mapping on your web.xml. See sample below how to declare the mapping.

  <mime-mapping>
        <extension>wsdl</extension>
        <mime-type>text/xml</mime-type>
  </mime-mapping>

Happy reading

GAE - Truncating Datastore Data

At this time there were no quick truncate for the Entities. Solution: Create data script in your servlet to delete.

import com.google.appengine.api.datastore.DatastoreService;
import com.google.appengine.api.datastore.DatastoreServiceFactory;
import com.google.appengine.api.datastore.Entity;
import com.google.appengine.api.datastore.PreparedQuery;
import com.google.appengine.api.datastore.Query;
....
@SuppressWarnings("serial")
public class GreetingServiceImpl extends RemoteServiceServlet implements
  GreetingService {

  protected void doGet(HttpServletRequest req,
    HttpServletResponse resp)
    throws ServletException, IOException
  {

    DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
      Query query = new Query("YourKind");
      PreparedQuery pq = datastore.prepare(query);
      // Iterate the date
      for (Entity result : pq.asIterable()) {
      Transaction txn = datastore.beginTransaction();
      try {
        datastore.delete(result.getKey());
        txn.commit();
      } finally {
        if (txn.isActive()) {
          txn.rollback();
        }
      } 
    }

    }// doGet

}// Class

GAE - Querying on the Datastore Entity with limit

Just share this simple statement in querying the Entity With limit. Assume the we want only specific number of report to be retrieve from our entity.

Query query = new Query(dataStoreKind);
PreparedQuery pq = datastore.prepare(query);
Iterable<Entity> users  = pq.asIterable(FetchOptions.Builder.withLimit(30));
for (Entity user : users) { // Scroll the data
  // user.getKey().getName(); // this how to get the Entity key
   // (Long)user.getProperty("EntityName"); // Get Entity value 

}

GAE - How to view the Datastore using eclipse

Thanks to. onpositive.com

GAE - How to filter on Datastore viewer using the Entity key or Primary

Search in you kind base record key

SELECT * FROM Kind WHERE __key__ = KEY('Kind', 'TheKeyValue')

Search on a kind base on the Parent key

SELECT * FROM Kind WHERE ANCESTOR IS KEY('ParentKind', 'ParentKeyWord')

Search all Entity base on Parent key what ever the Kind(See From Kind not exist)

SELECT * WHERE ANCESTOR IS KEY('ParentKind', 'ParentKeyWord')

More info hereGQL Syntax

GAE - Create Cron Job

For just one refence Appengine cron

GAE - Handing Custom Error Page

Java

<static-error-handlers>
  <handlerfile="default_error.html" />
  <handlerfile="over_quota.html"error-code="over_quota" />
</static-error-handlers>

Python

error_handlers:
 -file:default_error.html
 -error_code:over_quota
  file:over_quota.html

GAE - JSON implementation

Most of us also looking how to implement JSON on GAE. In one blog, posted how and he made it by create a POJO or maybe this was also called entity. Here's the impementation:

Class Response{
 public enum Status {
  ERROR, OK, WARNING
 }
 private Status status;
 private String message;

 public Response(Status status, String message)
 {
  this.status = status;
  this.message = message;
 }
 public String toJson() {
  Gson gson = new Gson();
  String json = gson.toJson(this);
  return json;
 }
 public static Response constructFromJson(String json)
 {
  Gson gson = new Gson();
  return gson.fromJson(json, Response.class);
 }
}

Here's how he implement:

public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException
{
 resp.setContentType("text/plain");
 //do your work here
 ...
 PrintWriter out = resp.getWriter();
 Response response = new Response(Response.Status.OK, "A success!");
 out.print(response.toJson());
}

Here's response looks:

{“status”:”OK”,”message”:”A success”}

TODO: Maybe someone ask. What if I want a array. Maybe try to looks at it and see how the implementation. This maybe we create a ArrayList properity or array it self. :) Give credit to nadacode.com

AppEngine - Samples

http://googcloudlabs.appspot.com/prepareforexercises.html

AppEngine - Guestbook sample

https://developers.google.com/cloud-sql/docs/developers_guide_java#using_the_java_development_server