k Nearest Neighbour in mysql

Question

I have the following table in MySQL:

DATE   EDGE   VALUE
D      E1       X1
D      E2       Y1
D      E3       Z1


D1      E1       X2
D1      E2       Y2
D1      E3       Z2


D2      E1       X3
D2     E2       Y3
D2      E3       Z3

Now I would like to calculate Euclidean distance of D to D1 and D to D2 like Distance(D-D1)= Sqrt((X1-X2)^2 +(Y1-Y2)^2 +(Z1-Z2)^2 ); Distance(D-D2)= Sqrt((X1-X3)^2 +(Y1-Y3)^2 +(Z1-Z3)^2 ); ....... and so on..

From this distance I want to choose 'k' nearest neighbors of D. (Please Note that It might be possible that record D might have any no of entries for edges(E1,E2...En). In this case other D1,D2,D3 will have same no of entries for edges...

Please suggest me solution as a stored procedure in MySQL...

Thanks in advance

---

@eggyal:

I tried to work on building similar query as answered by you.

Query:

SELECT   b.id,SQRT(SUM(POW(a.score - b.score, 2))) score1
FROM     (select * from data d1 where  d1.id = (select max(t1.id) from Timestamp t1) 
and d1.edge_id in (select m1.src_edge from mapping m1
where m1.dst = (select m2.src from mapping m2 where m2.src_edge=2 limit 1))) a
JOIN (select * from data d2 where d2.id in ( select t2.id from Timestamp t2 where DAYOFWEEK(NOW())=DAYOFWEEK(t2.timestamp)) and d2.edge_id in (select m3.src_edge from mapping m3 
where m3.dst = (select m4.src from mapping m4 
where m4.src_edge=2 limit 1))) as b
ON b.id <> a.id AND b.edge_id = a.edge_id 
GROUP BY b.id
ORDER BY score1
LIMIT    5;

However this is not good designed query. Please suggest all the improvements needed for the above query..

Thanks in advance

Table used for the above query: DATA table:

CREATE TABLE `data` (
  `id` bigint(20) DEFAULT NULL,
  `edge_id` bigint(20) NOT NULL,
  `score` int(11) NOT NULL,
  KEY `edge_id` (`edge_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
/*!50100 PARTITION BY RANGE (edge_id)
(PARTITION p0 VALUES LESS THAN (10000) ENGINE = InnoDB,
 PARTITION p1 VALUES LESS THAN (20000) ENGINE = InnoDB,
 PARTITION p2 VALUES LESS THAN (30000) ENGINE = InnoDB,
 PARTITION p3 VALUES LESS THAN (40000) ENGINE = InnoDB,
 PARTITION p4 VALUES LESS THAN (50000) ENGINE = InnoDB,
 PARTITION p5 VALUES LESS THAN (60000) ENGINE = InnoDB,
 PARTITION p6 VALUES LESS THAN (70000) ENGINE = InnoDB,
 PARTITION p7 VALUES LESS THAN (80000) ENGINE = InnoDB) */$$

Mapping table;

CREATE TABLE `mapping` (
  `dst` bigint(20) NOT NULL DEFAULT '0',
  `src` bigint(20) DEFAULT NULL,
  `src_edge` bigint(20) NOT NULL DEFAULT '0',
  PRIMARY KEY (`dst`,`src_edge`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1$$

Timestamp Table:

CREATE TABLE `Timestamp` (
  `timestamp` datetime NOT NULL,
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  PRIMARY KEY (`id`),
  KEY `time` (`timestamp`,`id`)
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=latin1$$

Answer 1

As mentioned in my comment above, why not simply perform a self-join and sort by the calculated distance?

SELECT   b.date
FROM     my_table a
    JOIN my_table b ON b.date <> a.date AND b.edge = a.edge
WHERE    a.date = ?
GROUP BY b.date
ORDER BY SUM(POW(a.value - b.value, 2))
LIMIT    ?

See it on sqlfiddle.

Answer 2

Maybe this can get you started...

CREATE TABLE nodes
(node_id CHAR(2) NOT NULL,   plane CHAR(1) NOT NULL, value INT NOT NULL, PRIMARY KEY(node_id,plane));

INSERT INTO nodes VALUES
('D','x',5),
('D','y',10),
('D','z',15),
('D1','x',20),
('D1','y',25),
('D1','z',30);

CREATE VIEW v_nodes AS 
SELECT node_id
    , MAX(CASE WHEN plane = 'x' THEN value END) x
    , MAX(CASE WHEN plane = 'y' THEN value END) y
    , MAX(CASE WHEN plane = 'z' THEN value END) z
 FROM nodes 
GROUP
   BY node_id;

SELECT ROUND(SQRT
                ( POW(ABS(d.x - d1.x),2) 
                + POW(ABS(d.y - d1.y),2)
                + POW(ABS(d.z - d1.z),2)
                )
           ,2)delta
  FROM v_nodes d
  JOIN v_nodes d1 
 WHERE d.node_id = 'd'
   AND d1.node_id = 'd1';

+-------+
| delta |
+-------+
| 25.98 |
+-------+   

Answer 3

This is the final solution:

SELECT   d2.id, SQRT(SUM(POW(d1.score - d2.score, 2))) score1
    FROM     data d1
        JOIN data d2 ON d2.id <> d1.id AND d2.edge_id = d1.edge_id
        JOIN mapping m1 ON m1.src_edge = d1.edge_id
        JOIN mapping m2 ON m2.src = m1.dst
        JOIN (SELECT MAX(t1.id) as id  FROM Timestamp) t1 ON t1.id = d1.id
        JOIN Timestamp t2 ON t2.id = d2.id
    WHERE    m2.src_edge = 2
         AND DAYOFWEEK(NOW()) = DAYOFWEEK(t2.timestamp)
    GROUP BY d2.id
    ORDER BY score1
    LIMIT    5;

This solution is given by eggyal.

I am completely agree with you eggyal that I should learn joins. Whatever solution I had proposed was my best. I will surely learn Joins asap. Thanks for your great help...