The just out of beta jquants-api propose sets of daily financial data from over 4000 listed companies on the Tokyo Stock Exchange.

Currently, the sets of data that can be accessed via the API are:

• Listed Issue Information
• Stock Price Information
• Market Information

with more to come in the near future.

While current historical data is only available starting from 2017 users are encouraged to provide feedback to the API via usual github (etc..) channels so as to understand usage in more details.

Building up on the previous iccube tutorial, here are a few key extensions to reporting. First how to extract data from a command line client written

Get data using CLI (or Java)

	String url, user, password, file;

    public static void main (String... args) throws Exception {
    
        Connection connection = DriverManager.getConnection(url, user, password);

        OlapConnection olapConnection = connection.unwrap(OlapConnection.class);
        runMDX(file, olapConnection);

        if (watch) {
            try {
                watch(file, olapConnection);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        }

        olapConnection.close();

    }


    public static String mdxQueryfromFile(String file) throws IOException {
        Path p = new File(file).toPath();
        System.out.println(p.toAbsolutePath());
        return Files.readString(p);
    }

    private static void runMDX(String file, OlapConnection olapConnection) throws OlapException, IOException {
        CellSet cellSet = olapConnection.createStatement().executeOlapQuery(mdxQueryfromFile(file));
        RectangularCellSetFormatter formatter = new RectangularCellSetFormatter(false);
        PrintWriter writer = new PrintWriter(System.out);
        formatter.format(cellSet, writer);
        writer.flush();
    }

Working with last N days/weeks etc…

WITH
SET LastWeek AS ClosingPeriod( [issueCompletion].[issueCompletion].[Day])
SET Last2Weeks AS {LastWeek.Item(0).Lag(13):LastWeek.Item(0)}
SELECT Last2Weeks ON 1, [Measures].[closedIssues] on 0
FROM [redmineProduction]

Create custom measures to compare with a previous period

create MEMBER [Measures].[OneYearAgo]
 AS
 (ParallelPeriod([Year],1),[Measures].[London Mean Roadside:Nitrogen Dioxide (ug/m3)])
   
create MEMBER [Measures].[DiffOneYearAgo]
 AS
Iif( IsEmpty( [Measures].[OneYearAgo] ), 0, [Measures].[London Mean Roadside:Nitrogen Dioxide (ug/m3)] - [Measures].[OneYearAgo] )

create MEMBER [Measures].[6 Months Moving Avg] AS
    AVG([Date].[Date].CurrentMember : [Date].[Date].CurrentMember.lag(6), [Measures].[London Mean Roadside:Nitrogen Dioxide (ug/m3)])

Export/Import server side + report

Find the schemas here …

Modrzyk, N. (2019). Clojure. In: Sakr, S., Zomaya, A.Y. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-77525-8_302

By the same sane community that gives us tablecloth, comes a full blown Machine Learning library named the scicloj.

Explaining the details of the whole library or machine learning in general being beyond the scope of this article, we will instead present how the reader can put into action a KNN, K Nearest Neighbor algorithm to guess quotes retrieved from the previously presented JQuants API. What is KNN? K Nearest Neighbor is a simple algorithm that stores all the available cases and classifies the new data or case based on a similarity measure. It is mostly used to classifies a data point based on how its neighbors are classified.

This has been re-published on dzone

Realtime React Coding in ClojureScript

I love ClojureScript. The language, the look, the feeling when I type the code to make my React components with it, makes me stay up all night. It actually did yesterday when I try to pick up on a friend setup and get back a modern environment to do my coding. Let’s walk through the first few steps to get to a setup with a React counter in ClojureScript, all this with live code reloading from VSCode.

Modrzyk, N. (2019). Clojure. In: Sakr, S., Zomaya, A.Y. (eds) Encyclopedia of Big Data Technologies. Springer, Cham. https://doi.org/10.1007/978-3-319-77525-8_302

In the Clojure world the Python’s numpy equivalent is named tablecloth, which acts as a data frame grammar wrapping tech.ml.dataset. It has much, if not all, the functions you would expect for data science associated with an impressive speed for handling data, provided you stick with the library functions. In this short section we will see how to work with tablecloth data frames, to handle data from the just out JQuants API from the land of the rising sun where sushi shines and the sky is blue.

With the OpenCV setup done before, it is now possible to go and actually use OpenCV for something practical. This is ported to Clojure from the following post, so please refer to it for the original write up.

The goal is to highlight different intensity of red circles in a picture.

The java imports when working with OpenCV are quite consistent between examples, so let’s add them along with the namespace definition.