When there is $\displaystyle s=\frac{3{x_1}^2-1}{2y_1}$ in the context of Elliptic Curve, that's computed in a field, that is a set with addition and multiplication such that usual rules of algebra apply, including a well-defined division by anything except $0$.
When further that's in the context of cryptography, the field is a finite field, noted $\mathbb F_q$, that has $q\ge2$ elements. Often, and in the context of the question and the rest of this answer, $q$ is a prime $p$, and then $\mathbb F_p$ is simply the integers modulo $p$, under addition and multiplication modulo $p$. When with $x,y,z\in\mathbb F_p$ we write some equality like $x=y\,z$, that really means this equality stands modulo $p$, that is $x\equiv y\,z\pmod p$, which by definition means: $p$ divides $y\,z-x$, where $x,y,z\in\mathbb Z$.
For example, with $p=17$ as in the question, $10\cdot4\equiv6\pmod p$ because $p=17$ divides $10\cdot4-6=34$, thus when the context stipulates we are computing in $\mathbb F_p$ (which I assume in the following) it holds $10\cdot4=6$.
Division works just the same as in any field like the familiar reals $\mathbb R$ or $\mathbb C$: the notation $\displaystyle z=\frac xy$ by definition means $y\ne0$ and $x=y\,z$. If follows $\displaystyle10=\frac 64$.
To make the calculation $\displaystyle z\gets\frac xy$, the first step is to check that $y\ne0$ (remember that's modulo $p$). Then for very small numbers like in the question, it's entirely feasible to search for $z$ until $x=y\,z$. It's also possible, and more common, to first compute $\displaystyle\frac 1y$, also noted $y^{-1}$, then compute $z\gets y^{-1}\,x$.
To compute $y^{-1}$ from $y\ne0$
- The textbook method is the Extended Euclidean Algorithm. Considering $y$ as an integer, we find integers $u$ and $v$ satisfying the Bezout equality $u\,p+v\,y=1$. It follows $v\bmod p$ is the desired $y^{-1}$. Since we do not need $u$, we need not compute it, and can use this. There are variants that work with non-negative integers at all steps, and others that need no modular reduction.
- Since $p$ is prime, by Fermat's little theorem, it holds $y^{p-1}=1$, thus $y^{-1}=y^{p-2}$. This tends to be slower, though.
It's pedagogically useful to implement the EEA, in particular because it works in any finite ring. However Python's pow has been extended so that pow(y, -1, p) can be used to compute the modular inverse of y modulo p with many Python 3.8 and later. With even more versions, we can get away with pow(y, p-2, p).
The question's ((3 * P[0] ** 2 + a)/ 2 * P[1]) % p has two issues:
- Most importantly,
/ is used for division, when that's the operator for (an approximation of) the division in $\mathbb R$, yielding a floating point value. Operator // would at least yield an integer, but when there's a rounding that would not be the correct result.
- As an aside, precedence of operators is such that
* P[1] is evaluated after and on the result of / 2, thus if that was not for the earlier issue, it would be computed $\displaystyle\frac{3{x_1}^2-1}2\,y_1$ rather than the desired $\displaystyle\frac{3{x_1}^2-1}{2y_1}$.
A correct expression is (3 * pow(P[0], 2, p) + a) * pow(2 * P[1], -1, p) % p. The first pow is to compute the square with reduction modulo $p$ (this is strictly optional). The second pow is to compute the modular inverse of the denominator.
There are other functional issues with the code, including the other computation of the slope and missing some cases; see the full addition algorithm in sec1 §3.2.2.
