In his article f'On the Drawing of Maps" P. L. Chebyshev beautifully expresses the nature of the relation between
scientific theory and practice (discussing the case of mathematics):
"The bringing together of theory and practice leads to the
most favorable' results; not only does practice benefit, but the
sciences themselves develop under the,influence of practice, which
reveals new subjects for investigation and new aspects of
familiar subjects." A striking example of the phenomenon
described by Chebyshev is afforded by the concept of entropy
in probability theory, a concept which has evolved in recent
years from the needs of practice. This concept first arose in
attempting to create a theoretical model for the transmission
of information of various kinds. In the beginning the concept
was introduced in intimate association with transmission
apparatus of one kind or another; its general theoretical
significance and properties, and the general nature of its application
to practice were only gradually realized. As of the present, a
unified exposition of the theory of entropy can be found only
in specialized articles and monographs dealing with the
transmission of information. Although the study of entropy has
actually evolved into an important and interesting chapter of
the general theory of probability, a presentation of it in this
general theoretical setting has so far been lacking.