From Newsgroup: comp.lang.prolog


Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986 https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History) https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog

Hi,

One idea I had was that autoencoders would
become kind of invisible, and work under the hood
to compress Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,0], [1,1,1,1,1,1,0]).
data(seg7, [0,1,1,0,0,0,0], [0,1,1,0,0,0,0]).
data(seg7, [1,1,0,1,1,0,1], [1,1,0,1,1,0,1]).
data(seg7, [1,1,1,1,0,0,1], [1,1,1,1,0,0,1]).
data(seg7, [0,1,1,0,0,1,1], [0,1,1,0,0,1,1]).
data(seg7, [1,0,1,1,0,1,1], [1,0,1,1,0,1,1]).
data(seg7, [1,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [1,1,1,0,0,0,0], [1,1,1,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,1], [1,1,1,1,1,1,1]).
data(seg7, [1,1,1,1,0,1,1], [1,1,1,1,0,1,1]).
% alternatives 9, 7, 6, 1
data(seg7, [1,1,1,0,0,1,1], [1,1,1,1,0,1,1]).
data(seg7, [1,1,1,0,0,1,0], [1,1,1,0,0,0,0]).
data(seg7, [0,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [0,0,0,0,1,1,0], [0,1,1,0,0,0,0]). https://en.wikipedia.org/wiki/Seven-segment_display

Or more visually, 9 7 6 1 have variants trained:

:- show.
_0123456789(9)(7)(6)(1)

The auto encoder would create a latent space, an
encoder, and a decoder. And we could basically query
?- data(seg7, X, Y) with X input, and Y output,

9 7 6 1 were corrected:

:- random2.
0, 0
_01234567899761

The autoencoder might also tolerate errors in the
input that are not in the data, giving it some inferential
capability. And then choose an output again not in

the data, giving it some generative capabilities.

Bye

See also:

What is Latent Space in Deep Learning? https://www.geeksforgeeks.org/what-is-latent-space-in-deep-learning/

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986 https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History) https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog

Hi,

Somebody wrote:

It’s a self-supervised form of ILP.
No autoencoders anywhere at all.

And, this only proofs my point that ILP doesn’t
solve the problem to make autoencoders and transformers
available directly in Prolog. Which was the issue I posted
at the top of this thread.

Subsequently I would not look into ILP for Prolog
autoencoders and transformers is my point exactly. Because
mostlikely ILP is unaware of the concept of latent space.
Latent space has quite some advantages:

- *Dimensionality Reduction:* It captures the essential
structure of high-dimensional data in a more
compact form.

- *Synthetic Data:* Instead of modifying raw data, you can
use the latent space, to generate variations for
further learning.

- *Domain Adaptation:* Well-structured latent space can
help transfer knowledge from abundant domains to
underrepresented ones.

If you don’t mention autoencoders and transformers at
all, you are possibly also not aware of the above advantages
and other properties of autoencoders and transformers.

In ILP mostlikely the concept of latent space is dormant
or blurred, since the stance is well we invent predicates,
ergo relations. There is no attempt to break

down relations further:

https://www.v7labs.com/blog/autoencoders-guide

Basically autoencoders and transformers, by imposing some
hidden layer, are further structuring relations into an
encoder and a decoder. So a relation is seen as a join.

The H is the bottleneck on purpose:

relation(X, Y) :- encoder(X, H), decoder(H, Y).

The values of H go through the latent space which is
invented during the learning process. It is not simply
the input or output space.

This design has some very interesting repercussions.

Bye

Mild Shock schrieb:

Hi,

One idea I had was that autoencoders would
become kind of invisible, and work under the hood
to compress Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,0], [1,1,1,1,1,1,0]).
data(seg7, [0,1,1,0,0,0,0], [0,1,1,0,0,0,0]).
data(seg7, [1,1,0,1,1,0,1], [1,1,0,1,1,0,1]).
data(seg7, [1,1,1,1,0,0,1], [1,1,1,1,0,0,1]).
data(seg7, [0,1,1,0,0,1,1], [0,1,1,0,0,1,1]).
data(seg7, [1,0,1,1,0,1,1], [1,0,1,1,0,1,1]).
data(seg7, [1,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [1,1,1,0,0,0,0], [1,1,1,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,1], [1,1,1,1,1,1,1]).
data(seg7, [1,1,1,1,0,1,1], [1,1,1,1,0,1,1]).
% alternatives 9, 7, 6, 1
data(seg7, [1,1,1,0,0,1,1], [1,1,1,1,0,1,1]).
data(seg7, [1,1,1,0,0,1,0], [1,1,1,0,0,0,0]).
data(seg7, [0,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [0,0,0,0,1,1,0], [0,1,1,0,0,0,0]). https://en.wikipedia.org/wiki/Seven-segment_display

Or more visually, 9 7 6 1 have variants trained:

:- show.
_0123456789(9)(7)(6)(1)

The auto encoder would create a latent space, an
encoder, and a decoder. And we could basically query
?- data(seg7, X, Y) with X input, and Y output,

9 7 6 1 were corrected:

:- random2.
0, 0
_01234567899761

The autoencoder might also tolerate errors in the
input that are not in the data, giving it some inferential
capability. And then choose an output again not in

the data, giving it some generative capabilities.

Bye

See also:

What is Latent Space in Deep Learning? https://www.geeksforgeeks.org/what-is-latent-space-in-deep-learning/

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986
https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History)
https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog


The problem I am trying to address was
already adressed here:

ILP and Reasoning by Analogy
Intuitively, the idea is to use what is already
known to explain new observations that appear similar
to old knowledge. In a sense, it is opposite of induction,
where to explain the observations one comes up with
new hypotheses/theories.
Vesna Poprcova et al. - 2010
https://www.researchgate.net/publication/220141214

The problem consists in that ILP doesn’t try to
learn and apply analogies , whereas autoencoders and
transformers typically try to “Grok” analogies, so that
with a fewer training they can perform

well in certain domains. They will do some inferencing
on the part of the encoders also for unseen input
data. And they will do some generation on the part of
the decoder also for unseen

latent space configurations from unseen input data.
By unseen data I mean data not in the training set.
The full context window may tune the inferencing and
generation, which appeals to:

Analogy as a Search Procedure
Rumelhart and Abrahamson showed that when presented
with analogy problems like mokey:pig:gorilla:X, with
rabbit, tiger, cow, and elephant as alternatives for X,
subjects rank the four options following the
parallelogram rule.
Matías Osta-Vélez - 2022
https://www.researchgate.net/publication/363700634

There are learning methods that work similarly
like ILP, in that they are based on positive and
negative samples. And the statistics can involve
bilinear forms, similar like

is seen in the “Attention is all you Need” paper.
But I have not yet a good implementation of this
evisioned marriage of autoencoders and ILP, and
I am still researching the topic.

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986 https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History) https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog

Hi,

I first wanted to use a working title:

"new frontiers in logic programming"

But upon reflection and because of fElon,
here another idea for a working title:

"neuro infused logic programming" (NILP)

What could it mean? Or does it have some
alternative phrasing already?

Try this paper:

Compositional Neural Logic Programming
Son N. Tran - 2021
The combination of connectionist models for low-level
information processing and logic programs for high-level
decision making can offer improvements in inference
efficiency and prediction performance https://www.ijcai.org/proceedings/2021/421

Browsing through the bibliography I find:

[Cohen et al., 2017]
Tensorlog: Deep learning meets probabilistic

[Donadello et al., 2017]
Logic tensor networks

[Larochelle and Murray, 2011]
The neural autoregressive distribution estimator

[Manhaeve et al., 2018]
Neural probabilistic logic programming

[Mirza and Osindero, 2014]
Conditional generative adversarial nets

[Odena et al., 2017]
auxiliary classifier GANs

[Pierrot et al., 2019]
compositional neural programs

[Reed and de Freitas, 2016]
Neural programmer-interpreters

[Riveret et al., 2020]
Neuro-Symbolic Probabilistic Argumentation Machines

[Serafini and d’Avila Garcez, 2016]
logic tensor networks.

[Socher et al., 2013]
neural tensor networks

[Towell and Shavlik, 1994]
Knowledge-based artificial neural networks

[Tran and d’Avila Garcez, 2018]
Deep logic networks

[Wang et al., 2019]
compositional neural information fusion


Mild Shock schrieb:

Hi,

One idea I had was that autoencoders would
become kind of invisible, and work under the hood
to compress Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,0], [1,1,1,1,1,1,0]).
data(seg7, [0,1,1,0,0,0,0], [0,1,1,0,0,0,0]).
data(seg7, [1,1,0,1,1,0,1], [1,1,0,1,1,0,1]).
data(seg7, [1,1,1,1,0,0,1], [1,1,1,1,0,0,1]).
data(seg7, [0,1,1,0,0,1,1], [0,1,1,0,0,1,1]).
data(seg7, [1,0,1,1,0,1,1], [1,0,1,1,0,1,1]).
data(seg7, [1,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [1,1,1,0,0,0,0], [1,1,1,0,0,0,0]).
data(seg7, [1,1,1,1,1,1,1], [1,1,1,1,1,1,1]).
data(seg7, [1,1,1,1,0,1,1], [1,1,1,1,0,1,1]).
% alternatives 9, 7, 6, 1
data(seg7, [1,1,1,0,0,1,1], [1,1,1,1,0,1,1]).
data(seg7, [1,1,1,0,0,1,0], [1,1,1,0,0,0,0]).
data(seg7, [0,0,1,1,1,1,1], [1,0,1,1,1,1,1]).
data(seg7, [0,0,0,0,1,1,0], [0,1,1,0,0,0,0]). https://en.wikipedia.org/wiki/Seven-segment_display

Or more visually, 9 7 6 1 have variants trained:

:- show.
_0123456789(9)(7)(6)(1)

The auto encoder would create a latent space, an
encoder, and a decoder. And we could basically query
?- data(seg7, X, Y) with X input, and Y output,

9 7 6 1 were corrected:

:- random2.
0, 0
_01234567899761

The autoencoder might also tolerate errors in the
input that are not in the data, giving it some inferential
capability. And then choose an output again not in

the data, giving it some generative capabilities.

Bye

See also:

What is Latent Space in Deep Learning? https://www.geeksforgeeks.org/what-is-latent-space-in-deep-learning/

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986
https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History)
https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog

Hi,

A software engineering analyis why Prolog fails ================================================

You would also get more done, if Prolog had some
well design plug and play machine learning libraries.
Currently most SWI Prolog packages are just GitHub dumps:

(Python) Problem ---> import solver ---> Solution

(SWI) Problem ---> install pack ---> Problem

Python shows more success in the practitioners domain,
since it has more libraries that have made the test of
time of practial use. Whereas Prolog is still in its
infancy in many domains,

you don’t arrive at the same level of convenience and
breadth as Python, if you have only fire and forget dumps
offered, from some PhD projects where software engineering
is secondary.

I don’t know exactly why Prolog has so much problems
with software engineering. Python has object orientation,
but Logtalk didn’t make the situation better. SWI-Prolog
has modules, but they are never used. For example this

here is a big monolith:

This module performs learning over Logic Programs https://github.com/friguzzi/liftcover/blob/main/prolog/liftcover.pl

Its more designed towards providing some command line
control. But if you look into it, it has EM algorithms
and gradient algorithm, and who knows what. These building
blocks are not exposed,

not made towards reused or towards improvement by
switching in 3rd party alternatives. Mostlikely a design
flaw inside the pack mechanism itself, since it assumes a
single main module?

So the pack mechanism works, if a unit pack imports a
clp(BNR) pack, since it uses the single entry of clp(BNR).
But it is never on paar with the richness of Python packages,
which have more a hierarchical structure of many

many modules in their packs.

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986 https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History) https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog

I have retracted those posts, that had Python-first
in it, not sure whether my analysis about some projects
was water thight. I only made the Python example as to
illustrate the idea of

a variation point. I do not think programming language
trench wars are good idea, and one should put software
engineering -first, as an abstract computer science
discipline. Not doing so

is only a distraction from the real issues at hand.
Variation points where defined quite vaguely
on purpose:

Ivar Jacobson defines a variation point as follows:
A variation point identifies one or more locations at
which the variation will occur.

Variation points can come in many shades, and for
example ProbLog based approaches take the viewpoint
of a Prolog text with a lot of configuration flags
and predicate

annotations. This is quite different from the
autoencoder or transformer component approach I
suggested here. In particular component oriented
approach could be

more flexible and dynamic, when they allow programmatic
configuration of components. The drawback is you cannot
understand what the program does by looking at a

simply structured Prolog text. Although I expected
the situation is not that bad, and one could do
something similar to a table/1 directive, i.e. some
directive that says

look, this predicate is an autoencoder or transformer:

One idea I had was that autoencoders would become
kind of invisible, and work under the hood to compress
Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).

So to instruct the Prolog system to do what is sketched,
one would possibly need a new directive autoencoder/1:

:- autoencoder data/3.

Mild Shock schrieb:

Hi,

A software engineering analyis why Prolog fails ================================================

You would also get more done, if Prolog had some
well design plug and play machine learning libraries.
Currently most SWI Prolog packages are just GitHub dumps:

(Python) Problem ---> import solver ---> Solution

(SWI) Problem ---> install pack ---> Problem

Python shows more success in the practitioners domain,
since it has more libraries that have made the test of
time of practial use. Whereas Prolog is still in its
infancy in many domains,

you don’t arrive at the same level of convenience and
breadth as Python, if you have only fire and forget dumps
offered, from some PhD projects where software engineering
is secondary.

I don’t know exactly why Prolog has so much problems
with software engineering. Python has object orientation,
but Logtalk didn’t make the situation better. SWI-Prolog
has modules, but they are never used. For example this

here is a big monolith:

This module performs learning over Logic Programs https://github.com/friguzzi/liftcover/blob/main/prolog/liftcover.pl

Its more designed towards providing some command line
control. But if you look into it, it has EM algorithms
and gradient algorithm, and who knows what. These building
blocks are not exposed,

not made towards reused or towards improvement by
switching in 3rd party alternatives. Mostlikely a design
flaw inside the pack mechanism itself, since it assumes a
single main module?

So the pack mechanism works, if a unit pack imports a
clp(BNR) pack, since it uses the single entry of clp(BNR).
But it is never on paar with the richness of Python packages,
which have more a hierarchical structure of many

many modules in their packs.

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986
https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf

Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History)
https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2

From Newsgroup: comp.lang.prolog


But even with such a directive there are many
challenges, which ProbLog suffers also from. Consider
this transformer pipeline, with two components of
type g twice:

+----+ +----+ +----+
| |-->| g |-->| |
| | +----+ | |
x -->| f | | h |--> y
| | +----+ | |
| |-->| g |-->| |
+----+ +----+ +----+

With common subexpessions, i.e. computing
f only once, I can write the forward pass
as follows:

p, q = f(x)
y = h(g(p), g(q))

But the above doesn’t show the learnt parameters.
Will g and g be siamese neural networks, learning
one sets of parameters, or will they learn
two sets of parameters? See also:

Siamese neural network
https://en.wikipedia.org/wiki/Siamese_neural_network

If I am not mistaken in ProbLog one can use
variables to indicate probabilities annotation.
An example of such a variable is seen here:

% intensional probabilistic fact with flexible probability:
P::pack(Item) :- weight(Item,Weight), P is 1.0/Weight.

But one might need something either to create
siamese or to separate siamese, depending on what
the default modus operandi of the probabilistic

logic programming language is.

Mild Shock schrieb:

I have retracted those posts, that had Python-first
in it, not sure whether my analysis about some projects
was water thight. I only made the Python example as to
illustrate the idea of

a variation point. I do not think programming language
trench wars are good idea, and one should put software
engineering -first, as an abstract computer science
discipline. Not doing so

is only a distraction from the real issues at hand.
Variation points where defined quite vaguely
on purpose:

Ivar Jacobson defines a variation point as follows:
A variation point identifies one or more locations at
which the variation will occur.

Variation points can come in many shades, and for
example ProbLog based approaches take the viewpoint
of a Prolog text with a lot of configuration flags
and predicate

annotations. This is quite different from the
autoencoder or transformer component approach I
suggested here. In particular component oriented
approach could be

more flexible and dynamic, when they allow programmatic
configuration of components. The drawback is you cannot
understand what the program does by looking at a

simply structured Prolog text. Although I expected
the situation is not that bad, and one could do
something similar to a table/1 directive, i.e. some
directive that says

look, this predicate is an autoencoder or transformer:

One idea I had was that autoencoders would become
kind of invisible, and work under the hood to compress
Prolog facts. Take these facts:

% standard _, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
data(seg7, [0,0,0,0,0,0,0], [0,0,0,0,0,0,0]).

So to instruct the Prolog system to do what is sketched,
one would possibly need a new directive autoencoder/1:

:- autoencoder data/3.

Mild Shock schrieb:

Hi,

A software engineering analyis why Prolog fails
================================================

You would also get more done, if Prolog had some
well design plug and play machine learning libraries.
Currently most SWI Prolog packages are just GitHub dumps:

(Python) Problem ---> import solver ---> Solution

(SWI) Problem ---> install pack ---> Problem

Python shows more success in the practitioners domain,
since it has more libraries that have made the test of
time of practial use. Whereas Prolog is still in its
infancy in many domains,

you don’t arrive at the same level of convenience and
breadth as Python, if you have only fire and forget dumps
offered, from some PhD projects where software engineering
is secondary.

I don’t know exactly why Prolog has so much problems
with software engineering. Python has object orientation,
but Logtalk didn’t make the situation better. SWI-Prolog
has modules, but they are never used. For example this

here is a big monolith:

This module performs learning over Logic Programs
https://github.com/friguzzi/liftcover/blob/main/prolog/liftcover.pl

Its more designed towards providing some command line
control. But if you look into it, it has EM algorithms
and gradient algorithm, and who knows what. These building
blocks are not exposed,

not made towards reused or towards improvement by
switching in 3rd party alternatives. Mostlikely a design
flaw inside the pack mechanism itself, since it assumes a
single main module?

So the pack mechanism works, if a unit pack imports a
clp(BNR) pack, since it uses the single entry of clp(BNR).
But it is never on paar with the richness of Python packages,
which have more a hierarchical structure of many

many modules in their packs.

Mild Shock schrieb:

Inductive logic programming at 30
https://arxiv.org/abs/2102.10556

The paper contains not a single reference to autoencoders!
Still they show this example:

Fig. 1 ILP systems struggle with structured examples that
exhibit observational noise. All three examples clearly
spell the word "ILP", with some alterations: 3 noisy pixels,
shifted and elongated letters. If we would be to learn a
program that simply draws "ILP" in the middle of the picture,
without noisy pixels and elongated letters, that would
be a correct program.

I guess ILP is 30 years behind the AI boom. An early autoencoder
turned into transformer was already reported here (*):

SERIAL ORDER, Michael I. Jordan - May 1986
https://cseweb.ucsd.edu/~gary/PAPER-SUGGESTIONS/Jordan-TR-8604-OCRed.pdf >>>
Well ILP might have its merits, maybe we should not ask
for a marriage of LLM and Prolog, but Autoencoders and ILP.
But its tricky, I am still trying to decode the da Vinci code of

things like stacked tensors, are they related to k-literal clauses?
The paper I referenced is found in this excellent video:

The Making of ChatGPT (35 Year History)
https://www.youtube.com/watch?v=OFS90-FX6pg

--- Synchronet 3.20c-Linux NewsLink 1.2