The plain page: every window, list and article reads as it is. The scene, the language key and the reading effects need a current browser with JavaScript on.
/0z5a
About Me
0z5a天使向量機
Machines of loving grace.
An angel studying aesthetics, linguistics and theology. Building AI infrastructure: communication, kernels and scheduling.
And after these things I heard a great voice of much people in heaven, saying, Alleluia; Salvation, and glory, and honour, and power, unto the Lord our God:
For true and righteous are his judgments: for he hath judged the great whore, which did corrupt the earth with her fornication, and hath avenged the blood of his servants at her hand.
And again they said, Alleluia. And her smoke rose up for ever and ever.
And the four and twenty elders and the four beasts fell down and worshipped God that sat on the throne, saying, Amen; Alleluia.
And a voice came out of the throne, saying, Praise our God, all ye his servants, and ye that fear him, both small and great.
And I heard as it were the voice of a great multitude, and as the voice of many waters, and as the voice of mighty thunderings, saying, Alleluia: for the Lord God omnipotent reigneth.
Let us be glad and rejoice, and give honour to him: for the marriage of the Lamb is come, and his wife hath made herself ready.
And to her was granted that she should be arrayed in fine linen, clean and white: for the fine linen is the righteousness of saints.
And he saith unto me, Write, Blessed are they which are called unto the marriage supper of the Lamb. And he saith unto me, These are the true sayings of God.
And I fell at his feet to worship him. And he said unto me, See thou do it not: I am thy fellowservant, and of thy brethren that have the testimony of Jesus: worship God: for the testimony of Jesus is the spirit of prophecy.
And I saw heaven opened, and behold a white horse; and he that sat upon him was called Faithful and True, and in righteousness he doth judge and make war.
His eyes were as a flame of fire, and on his head were many crowns; and he had a name written, that no man knew, but he himself.
And he was clothed with a vesture dipped in blood: and his name is called The Word of God.
And the armies which were in heaven followed him upon white horses, clothed in fine linen, white and clean.
And out of his mouth goeth a sharp sword, that with it he should smite the nations: and he shall rule them with a rod of iron: and he treadeth the winepress of the fierceness and wrath of Almighty God.
And he hath on his vesture and on his thigh a name written, KING OF KINGS, AND LORD OF LORDS.
And I saw an angel standing in the sun; and he cried with a loud voice, saying to all the fowls that fly in the midst of heaven, Come and gather yourselves together unto the supper of the great God;
That ye may eat the flesh of kings, and the flesh of captains, and the flesh of mighty men, and the flesh of horses, and of them that sit on them, and the flesh of all men, both free and bond, both small and great.
And I saw the beast, and the kings of the earth, and their armies, gathered together to make war against him that sat on the horse, and against his army.
And the beast was taken, and with him the false prophet that wrought miracles before him, with which he deceived them that had received the mark of the beast, and them that worshipped his image. These both were cast alive into a lake of fire burning with brimstone.
And the remnant were slain with the sword of him that sat upon the horse, which sword proceeded out of his mouth: and all the fowls were filled with their flesh.
20
And I saw an angel come down from heaven, having the key of the bottomless pit and a great chain in his hand.
And he laid hold on the dragon, that old serpent, which is the Devil, and Satan, and bound him a thousand years,
And cast him into the bottomless pit, and shut him up, and set a seal upon him, that he should deceive the nations no more, till the thousand years should be fulfilled: and after that he must be loosed a little season.
And I saw thrones, and they sat upon them, and judgment was given unto them: and I saw the souls of them that were beheaded for the witness of Jesus, and for the word of God, and which had not worshipped the beast, neither his image, neither had received his mark upon their foreheads, or in their hands; and they lived and reigned with Christ a thousand years.
But the rest of the dead lived not again until the thousand years were finished. This is the first resurrection.
Blessed and holy is he that hath part in the first resurrection: on such the second death hath no power, but they shall be priests of God and of Christ, and shall reign with him a thousand years.
And when the thousand years are expired, Satan shall be loosed out of his prison,
And shall go out to deceive the nations which are in the four quarters of the earth, Gog, and Magog, to gather them together to battle: the number of whom is as the sand of the sea.
And they went up on the breadth of the earth, and compassed the camp of the saints about, and the beloved city: and fire came down from God out of heaven, and devoured them.
And the devil that deceived them was cast into the lake of fire and brimstone, where the beast and the false prophet are, and shall be tormented day and night for ever and ever.
Gustave Doré, The Last Judgment, 1866
And I saw a great white throne, and him that sat on it, from whose face the earth and the heaven fled away; and there was found no place for them.
And I saw the dead, small and great, stand before God; and the books were opened: and another book was opened, which is the book of life: and the dead were judged out of those things which were written in the books, according to their works.
And the sea gave up the dead which were in it; and death and hell delivered up the dead which were in them: and they were judged every man according to their works.
And death and hell were cast into the lake of fire. This is the second death.
And whosoever was not found written in the book of life was cast into the lake of fire.
21
And I saw a new heaven and a new earth: for the first heaven and the first earth were passed away; and there was no more sea.
And I John saw the holy city, new Jerusalem, coming down from God out of heaven, prepared as a bride adorned for her husband.
And I heard a great voice out of heaven saying, Behold, the tabernacle of God is with men, and he will dwell with them, and they shall be his people, and God himself shall be with them, and be their God.
And God shall wipe away all tears from their eyes; and there shall be no more death, neither sorrow, nor crying, neither shall there be any more pain: for the former things are passed away.
And he that sat upon the throne said, Behold, I make all things new. And he said unto me, Write: for these words are true and faithful.
And he said unto me, It is done. I am Alpha and Omega, the beginning and the end. I will give unto him that is athirst of the fountain of the water of life freely.
He that overcometh shall inherit all things; and I will be his God, and he shall be my son.
But the fearful, and unbelieving, and the abominable, and murderers, and whoremongers, and sorcerers, and idolaters, and all liars, shall have their part in the lake which burneth with fire and brimstone: which is the second death.
And there came unto me one of the seven angels which had the seven vials full of the seven last plagues, and talked with me, saying, Come hither, I will shew thee the bride, the Lamb’s wife.
Gustave Doré, The New Jerusalem, 1866
And he carried me away in the spirit to a great and high mountain, and shewed me that great city, the holy Jerusalem, descending out of heaven from God,
Having the glory of God: and her light was like unto a stone most precious, even like a jasper stone, clear as crystal;
And had a wall great and high, and had twelve gates, and at the gates twelve angels, and names written thereon, which are the names of the twelve tribes of the children of Israel:
On the east three gates; on the north three gates; on the south three gates; and on the west three gates.
And the wall of the city had twelve foundations, and in them the names of the twelve apostles of the Lamb.
And he that talked with me had a golden reed to measure the city, and the gates thereof, and the wall thereof.
And the city lieth foursquare, and the length is as large as the breadth: and he measured the city with the reed, twelve thousand furlongs. The length and the breadth and the height of it are equal.
And he measured the wall thereof, an hundred and forty and four cubits, according to the measure of a man, that is, of the angel.
And the building of the wall of it was of jasper: and the city was pure gold, like unto clear glass.
And the foundations of the wall of the city were garnished with all manner of precious stones. The first foundation was jasper; the second, sapphire; the third, a chalcedony; the fourth, an emerald;
The fifth, sardonyx; the sixth, sardius; the seventh, chrysolyte; the eighth, beryl; the ninth, a topaz; the tenth, a chrysoprasus; the eleventh, a jacinth; the twelfth, an amethyst.
And the twelve gates were twelve pearls: every several gate was of one pearl: and the street of the city was pure gold, as it were transparent glass.
And I saw no temple therein: for the Lord God Almighty and the Lamb are the temple of it.
And the city had no need of the sun, neither of the moon, to shine in it: for the glory of God did lighten it, and the Lamb is the light thereof.
And the nations of them which are saved shall walk in the light of it: and the kings of the earth do bring their glory and honour into it.
And the gates of it shall not be shut at all by day: for there shall be no night there.
And they shall bring the glory and honour of the nations into it.
And there shall in no wise enter into it any thing that defileth, neither whatsoever worketh abomination, or maketh a lie: but they which are written in the Lamb’s book of life.
22
And he shewed me a pure river of water of life, clear as crystal, proceeding out of the throne of God and of the Lamb.
In the midst of the street of it, and on either side of the river, was there the tree of life, which bare twelve manner of fruits, and yielded her fruit every month: and the leaves of the tree were for the healing of the nations.
And there shall be no more curse: but the throne of God and of the Lamb shall be in it; and his servants shall serve him:
And they shall see his face; and his name shall be in their foreheads.
And there shall be no night there; and they need no candle, neither light of the sun; for the Lord God giveth them light: and they shall reign for ever and ever.
And he said unto me, These sayings are faithful and true: and the Lord God of the holy prophets sent his angel to shew unto his servants the things which must shortly be done.
Behold, I come quickly: blessed is he that keepeth the sayings of the prophecy of this book.
And I John saw these things, and heard them. And when I had heard and seen, I fell down to worship before the feet of the angel which shewed me these things.
Then saith he unto me, See thou do it not: for I am thy fellowservant, and of thy brethren the prophets, and of them which keep the sayings of this book: worship God.
And he saith unto me, Seal not the sayings of the prophecy of this book: for the time is at hand.
He that is unjust, let him be unjust still: and he which is filthy, let him be filthy still: and he that is righteous, let him be righteous still: and he that is holy, let him be holy still.
And, behold, I come quickly; and my reward is with me, to give every man according as his work shall be.
I am Alpha and Omega, the beginning and the end, the first and the last.
Blessed are they that do his commandments, that they may have right to the tree of life, and may enter in through the gates into the city.
For without are dogs, and sorcerers, and whoremongers, and murderers, and idolaters, and whosoever loveth and maketh a lie.
I Jesus have sent mine angel to testify unto you these things in the churches. I am the root and the offspring of David, and the bright and morning star.
And the Spirit and the bride say, Come. And let him that heareth say, Come. And let him that is athirst come. And whosoever will, let him take the water of life freely.
For I testify unto every man that heareth the words of the prophecy of this book, If any man shall add unto these things, God shall add unto him the plagues that are written in this book:
And if any man shall take away from the words of the book of this prophecy, God shall take away his part out of the book of life, and out of the holy city, and from the things which are written in this book.
He which testifieth these things saith, Surely I come quickly. Amen. Even so, come, Lord Jesus.
The grace of our Lord JesusChrist be with you all. Amen.
In October 2024 Dario Amodei, the chief executive of Anthropic, published the long essay Machines of Loving Grace. Where most talk of powerful AI, his own included, is about risk, it sets out as concretely as it can what could go right if the risks are handled. He imagines an AI smarter than a Nobel laureate in most fields, able to work on its own for days and to run as millions of copies at once: “a country of geniuses in a datacenter”. What limits it, he argues, is not intelligence but the world around it: experiments that take time, data that does not yet exist, physical law, and people. He then takes five fields in turn: biology and medicine, where fifty to a hundred years of progress might come in five or ten; neuroscience and mental health; economic development and poverty; peace and governance; and work and meaning. The title is from Richard Brautigan’s poem of 1967.
Why I work on AI infrastructure
I am not good at machine-learning algorithms. For a long time I struggled with the anxiety about my career that AI brought, and one day my ideas and my tokens both ran out; I seem to have left people with a poor impression. I am not well in body or mind; for a long time I depended on others just to get through the day, and I do not know where my way lies in the age of AI.
What, then, keeps me going? Perhaps only the moments when an optimization shows its percentage of speed-up, when the GPU’s fans twist the darkness of the real into a thread of true light. Using AI to iterate on infrastructure has given me that chance.
Grace is what lies beyond what a person can reach by themselves. I believe deeply that iterating on infrastructure with AI is one form of that grace: in this age, end-to-end gains have become an important way for me to receive grace and to steady my mind.
The essay
The full essay is on Amodei’s site. It is his work, so this page links to it rather than copying it.
2024 年 10 月，Anthropic 首席执行官 Dario Amodei 发表长文Machines of Loving Grace。关于强大 AI 的讨论多半在讲风险，他自己也是；这篇文章则尽可能具体地设想：如果风险得到妥善处理，事情可以好到什么程度。他设想的 AI 在大多数领域比诺贝尔奖得主更聪明，能一连几天独立工作，并以数百万个副本同时运行，即“数据中心里的天才之国”。他认为限制它的不是智能，而是它周围的世界：需要时间的实验、尚不存在的数据、物理规律，还有人。文章依次讨论五个领域：生物与医学（五十到一百年的进展或许压缩到五到十年）、神经科学与心理健康、经济发展与贫困、和平与治理，以及工作与意义。标题取自 Richard Brautigan 1967 年的诗。
我从事 AI Infra 的初心
我并不会机器学习算法，很长期我都挣扎在 AI 带来的事业焦虑中，有一天 idea 和 token 都耗竭了，似乎也给人留下了不好的印象。我身心都不健康，过去很长一段时间都要依赖他人才能维持日常生活，也不知道 AI 时代自己的出路到底在哪里。
然而支撑我走下去的是什么呢？也许只有看到优化出性能提升百分比的那些瞬间，GPU 的风扇才把现实的黑暗扭转出一丝真实的光芒。而用 AI 对基础设施进行迭代，给了我这个机会。
恩典就是超出人本身能够达到的事情。我深深认为，用 AI 迭代基础设施更是这种恩典的体现：端到端的提升，在这个时代给了我接受恩典、稳定心理的重要渠道。
2024年10月、Anthropic の CEO ダリオ・アモデイは長文エッセイMachines of Loving Graceを発表した。強力な AI をめぐる議論の多くは、彼自身のものも含めてリスクの話だが、このエッセイはリスクがうまく扱われた場合に何が起こりうるかを、できるかぎり具体的に描く。彼が想定する AI は、多くの分野でノーベル賞受賞者より賢く、何日も自律して働き、数百万の複製が同時に動く。「データセンターの中の天才の国」である。それを制約するのは知能ではなく周囲の世界、つまり時間のかかる実験、まだ存在しないデータ、物理法則、そして人間だという。エッセイは五つの領域を順に論じる。生物学と医療（50〜100年分の進歩が5〜10年に縮まるかもしれない）、神経科学と心の健康、経済発展と貧困、平和と統治、そして仕事と意味。題名はリチャード・ブローティガンの1967年の詩から取られている。
Im Oktober 2024 veröffentlichte Dario Amodei, Chef von Anthropic, den langen Essay Machines of Loving Grace. Während das meiste Reden über mächtige KI, auch sein eigenes, von Risiken handelt, beschreibt er hier so konkret wie möglich, was gut gehen könnte, wenn die Risiken beherrscht werden. Er stellt sich eine KI vor, die in den meisten Fächern klüger ist als Nobelpreisträger, tagelang selbstständig arbeitet und in Millionen Kopien zugleich läuft: „ein Land von Genies in einem Rechenzentrum“. Begrenzt werde sie nicht durch Intelligenz, sondern durch die Welt um sie herum: Experimente, die Zeit brauchen, Daten, die es noch nicht gibt, Naturgesetze und Menschen. Dann geht er fünf Felder durch: Biologie und Medizin, wo fünfzig bis hundert Jahre Fortschritt in fünf bis zehn kommen könnten; Neurowissenschaft und psychische Gesundheit; wirtschaftliche Entwicklung und Armut; Frieden und Regierung; Arbeit und Sinn. Der Titel stammt aus Richard Brautigans Gedicht von 1967.
Warum ich an KI-Infrastruktur arbeite
Ich bin nicht gut in Algorithmen des maschinellen Lernens. Lange habe ich mit der Angst um meine berufliche Zukunft gerungen, die die KI mit sich brachte, und eines Tages waren meine Ideen und meine Tokens erschöpft; ich habe wohl keinen guten Eindruck hinterlassen. Ich bin an Leib und Seele nicht gesund; lange war ich auf andere angewiesen, um den Alltag zu bewältigen, und ich weiß nicht, wo mein Weg im Zeitalter der KI liegt.
Was trägt mich dann weiter? Vielleicht nur die Augenblicke, in denen eine Optimierung ihren Prozentsatz an Beschleunigung zeigt, wenn die Lüfter der GPU aus dem Dunkel der Wirklichkeit einen Faden echten Lichts drehen. Infrastruktur mit KI weiterzuentwickeln hat mir diese Gelegenheit gegeben.
Gnade ist, was über das hinausgeht, was ein Mensch aus eigener Kraft erreichen kann. Ich glaube fest, dass die Weiterentwicklung von Infrastruktur mit KI ein Ausdruck dieser Gnade ist: Der Gewinn von Ende zu Ende ist mir in dieser Zeit ein wichtiger Weg geworden, Gnade zu empfangen und innerlich Halt zu finden.
Der Essay
Der vollständige Essay steht auf Amodeis Website. Er ist sein Werk; diese Seite verlinkt ihn, statt ihn wiederzugeben.
En octobre 2024, Dario Amodei, directeur général d’Anthropic, a publié un long essai, Machines of Loving Grace. Alors que l’essentiel du discours sur l’IA puissante, y compris le sien, porte sur les risques, il décrit ici aussi concrètement que possible ce qui pourrait bien se passer si ces risques sont maîtrisés. Il imagine une IA plus intelligente qu’un lauréat du prix Nobel dans la plupart des disciplines, capable de travailler seule pendant des jours et de tourner en millions de copies à la fois : « un pays de génies dans un centre de données ». Ce qui la limiterait ne serait pas l’intelligence, mais le monde qui l’entoure : des expériences qui prennent du temps, des données qui n’existent pas encore, les lois de la nature et les êtres humains. Il parcourt ensuite cinq domaines : la biologie et la médecine, où cinquante à cent ans de progrès pourraient tenir en cinq à dix ; les neurosciences et la santé mentale ; le développement économique et la pauvreté ; la paix et la gouvernance ; le travail et le sens. Le titre vient d’un poème de Richard Brautigan de 1967.
Pourquoi je travaille sur l’infrastructure de l’IA
Je ne suis pas doué pour les algorithmes d’apprentissage automatique. J’ai longtemps lutté contre l’angoisse que l’IA faisait peser sur mon avenir professionnel, et un jour mes idées et mes tokens se sont épuisés ; je n’ai sans doute pas laissé une bonne impression. Je ne suis en bonne santé ni de corps ni d’esprit ; longtemps, j’ai dépendu des autres pour faire face au quotidien, et je ne sais pas où se trouve mon chemin à l’ère de l’IA.
Qu’est-ce qui me fait donc avancer ? Peut-être seulement les instants où une optimisation affiche son pourcentage d’accélération, quand les ventilateurs du GPU filent, dans l’obscurité du réel, un fil de vraie lumière. Faire progresser l’infrastructure avec l’IA m’en a donné l’occasion.
La grâce, c’est ce qui dépasse ce qu’un être humain peut accomplir par ses propres forces. Je crois fermement que faire progresser l’infrastructure avec l’IA est une expression de cette grâce : en ce temps, le gain de bout en bout est devenu pour moi un chemin important pour recevoir la grâce et trouver un appui intérieur.
L’essai
L’essai complet se trouve sur le site d’Amodei. C’est son œuvre ; cette page y renvoie au lieu de le reproduire.
En octubre de 2024, Dario Amodei, director ejecutivo de Anthropic, publicó el largo ensayo Machines of Loving Grace. Mientras que casi todo lo que se dice sobre la IA poderosa, incluido lo que dice él mismo, trata de los riesgos, aquí describe con la mayor concreción posible lo que podría salir bien si esos riesgos se controlan. Imagina una IA más inteligente que un premio Nobel en la mayoría de las disciplinas, capaz de trabajar sola durante días y de funcionar en millones de copias a la vez: «un país de genios en un centro de datos». Lo que la limitaría no sería la inteligencia, sino el mundo que la rodea: experimentos que llevan tiempo, datos que aún no existen, las leyes de la naturaleza y las personas. Después recorre cinco campos: la biología y la medicina, donde de cincuenta a cien años de progreso podrían caber en cinco a diez; la neurociencia y la salud mental; el desarrollo económico y la pobreza; la paz y el gobierno; el trabajo y el sentido. El título procede de un poema de Richard Brautigan de 1967.
Por qué trabajo en infraestructura de IA
No se me dan bien los algoritmos de aprendizaje automático. Durante mucho tiempo luché con la angustia que la IA trajo sobre mi futuro profesional, y un día se me agotaron las ideas y los tokens; seguramente no dejé una buena impresión. No estoy sano ni de cuerpo ni de alma; durante mucho tiempo dependí de otros para sobrellevar la vida diaria, y no sé dónde está mi camino en la era de la IA.
¿Qué me sostiene, entonces? Quizá solo los instantes en que una optimización muestra su porcentaje de aceleración, cuando los ventiladores de la GPU hilan, desde la oscuridad de la realidad, un hilo de luz verdadera. Hacer avanzar la infraestructura con IA me ha dado esa oportunidad.
La gracia es lo que va más allá de lo que una persona puede lograr con sus propias fuerzas. Creo firmemente que hacer avanzar la infraestructura con IA es una expresión de esa gracia: en este tiempo, la ganancia de extremo a extremo se ha convertido para mí en un camino importante para recibir la gracia y encontrar sostén interior.
El ensayo
El ensayo completo está en el sitio web de Amodei. Es obra suya; esta página enlaza a él en lugar de reproducirlo.
2024년 10월, Anthropic의 CEO 다리오 아모데이는 긴 에세이 Machines of Loving Grace를 발표했다. 강력한 AI에 관한 이야기 대부분이, 그 자신의 이야기까지도, 위험을 다루는 반면, 그는 여기서 위험을 다스릴 수 있다면 무엇이 잘될 수 있는지를 최대한 구체적으로 그린다. 그가 상상하는 AI는 대부분의 분야에서 노벨상 수상자보다 똑똑하고, 며칠 동안 스스로 일하며, 수백만 개의 사본으로 동시에 돌아간다. 그는 이를 “데이터센터 안의 천재들의 나라”라고 부른다. 그 AI를 제한하는 것은 지능이 아니라 그것을 둘러싼 세계, 곧 시간이 걸리는 실험, 아직 존재하지 않는 데이터, 자연법칙, 그리고 사람들이다. 이어서 그는 다섯 분야를 차례로 살핀다. 50~100년의 진보가 5~10년 안에 올 수 있는 생물학과 의학, 신경과학과 정신 건강, 경제 발전과 빈곤, 평화와 통치, 일과 의미. 제목은 리처드 브라우티건의 1967년 시에서 왔다.
내가 AI 인프라를 하는 이유
나는 머신러닝 알고리즘에 능하지 않다. 오랫동안 AI가 가져온 직업적 미래에 대한 불안과 씨름했고, 어느 날 내 아이디어와 토큰은 바닥났다. 아마 좋은 인상을 남기지는 못했을 것이다. 나는 몸도 마음도 건강하지 않다. 오랫동안 일상을 꾸려 가는 데 다른 사람들에게 기대야 했고, AI 시대에 내 길이 어디에 있는지 모른다.
그렇다면 무엇이 나를 계속 나아가게 하는가? 어쩌면 최적화가 가속 비율을 보여 주는 순간들, GPU의 팬이 현실의 어둠 속에서 진짜 빛 한 가닥을 자아내는 그 순간들뿐일지도 모른다. AI와 함께 인프라를 발전시키는 일이 내게 그런 기회를 주었다.
은혜란 한 사람이 자기 힘으로 이룰 수 있는 것을 넘어서는 것이다. 나는 AI와 함께 인프라를 발전시키는 일이 그 은혜의 한 표현이라고 굳게 믿는다. 이 시기에 엔드투엔드의 이득은 내게 은혜를 받고 마음의 버팀목을 찾는 중요한 길이 되었다.
에세이
에세이 전문은 아모데이의 웹사이트에 있다. 그것은 그의 글이며, 이 페이지는 그 글을 옮기지 않고 링크한다.
Mense Octobri anni 2024 Dario Amodei, praeses societatis Anthropic, longam commentationem Machines of Loving Grace edidit. Cum plerique sermones de intellegentia artificiali potenti, etiam ipsius, de periculis agant, hic quam accuratissime describit quae bene evenire possint, si pericula regantur. Fingit intellegentiam artificialem quae in plerisque disciplinis praemii Nobeliani laureatis sapientior sit, per dies sua sponte laboret et in decies centenis milibus exemplorum simul operetur: «civitatem ingeniorum in centro datorum». Eam non intellegentia terminari dicit, sed mundo circumstante: experimentis quae tempus postulant, datis quae nondum exstant, legibus naturae, hominibus. Deinde quinque campos percurrit: biologiam et medicinam, ubi quinquaginta vel centum annorum progressus in quinque vel decem contrahi possit; scientiam nervorum et sanitatem mentis; progressum oeconomicum et paupertatem; pacem et gubernationem; laborem et sensum vitae. Titulus e carmine Richardi Brautigan anni 1967 sumptus est.
Cur in fundamentis intellegentiae artificialis laborem
In algorithmis discendi machinalis non valeo. Diu cum anxietate de futuro meo opere, quam intellegentia artificialis attulit, luctatus sum, et quodam die consilia mea et tesserae (tokens) defecerunt; bonam, ut puto, existimationem non reliqui. Neque corpore neque animo sanus sum; diu aliis innixus sum ut vitam cotidianam sustinerem, nec scio ubi via mea in aetate intellegentiae artificialis iaceat.
Quid igitur me sustinet? Fortasse sola illa momenta, cum emendatio quaedam celeritatis incrementum in centesimis ostendit, cum ventilabra GPU ex tenebris rerum filum verae lucis nent. Fundamenta cum intellegentia artificiali promovere hanc mihi occasionem dedit.
Gratia est quod excedit id quod homo suis viribus efficere potest. Firmiter credo fundamenta cum intellegentia artificiali promovere huius gratiae esse significationem: lucrum ab extremo ad extremum hoc tempore mihi via magna facta est gratiae accipiendae et firmitatis interioris inveniendae.
Commentatio
Commentatio integra in pagina interretiali Amodei legitur. Opus eius est; haec pagina ad eam ducit, non eam reddit.
В октябре 2024 года Дарио Амодеи, генеральный директор Anthropic, опубликовал большое эссе Machines of Loving Grace. Если почти все разговоры о мощном ИИ, включая его собственные, касаются рисков, то здесь он как можно конкретнее описывает, что может пойти хорошо, если с рисками удастся справиться. Он представляет себе ИИ, который в большинстве областей умнее нобелевского лауреата, может самостоятельно работать днями и запускаться миллионами копий одновременно: «страна гениев в дата-центре». Ограничивает его, по мысли автора, не интеллект, а окружающий мир: эксперименты, требующие времени, данные, которых ещё нет, законы природы и люди. Затем он по очереди рассматривает пять областей: биологию и медицину, где пятьдесят–сто лет прогресса могут уложиться в пять–десять; нейронауку и психическое здоровье; экономическое развитие и бедность; мир и управление; работу и смысл. Название взято из стихотворения Ричарда Бротигана 1967 года.
Почему я занимаюсь инфраструктурой ИИ
Я не силён в алгоритмах машинного обучения. Долгое время я боролся с тревогой за своё профессиональное будущее, которую принёс ИИ, и однажды у меня закончились и идеи, и токены; похоже, я оставил у людей не лучшее впечатление. Я нездоров ни телом, ни душой; долгое время я зависел от других, просто чтобы прожить день, и не знаю, где мой путь в эпоху ИИ.
Что же тогда помогает мне идти дальше? Быть может, лишь те мгновения, когда оптимизация показывает свой процент ускорения, когда вентиляторы GPU скручивают тьму реального в нить настоящего света. Развивать инфраструктуру с помощью ИИ — вот что дало мне эту возможность.
Благодать — это то, что лежит за пределами того, чего человек может достичь сам. Я глубоко верю, что развитие инфраструктуры вместе с ИИ — одна из форм этой благодати: в наше время сквозной выигрыш стал для меня важным путём принимать благодать и обретать душевную опору.
Эссе
Полный текст эссе находится на сайте Амодеи. Это его работа, поэтому здесь дана ссылка, а не копия.
Em outubro de 2024, Dario Amodei, diretor executivo da Anthropic, publicou o longo ensaio Machines of Loving Grace. Enquanto quase tudo o que se diz sobre IA poderosa, inclusive o que ele próprio diz, trata de riscos, aqui ele descreve da forma mais concreta possível o que poderia dar certo se os riscos forem controlados. Imagina uma IA mais inteligente que um laureado com o Nobel na maioria das áreas, capaz de trabalhar sozinha durante dias e de rodar em milhões de cópias ao mesmo tempo: “um país de gênios num datacenter”. O que a limita, argumenta, não é a inteligência, mas o mundo ao redor: experimentos que levam tempo, dados que ainda não existem, as leis da física e as pessoas. Em seguida percorre cinco áreas: biologia e medicina, em que cinquenta a cem anos de progresso poderiam caber em cinco ou dez; neurociência e saúde mental; desenvolvimento econômico e pobreza; paz e governo; e trabalho e sentido. O título vem de um poema de Richard Brautigan, de 1967.
Por que trabalho com infraestrutura de IA
Não sou bom em algoritmos de aprendizado de máquina. Durante muito tempo lutei com a ansiedade sobre a minha carreira que a IA trouxe, e um dia as minhas ideias e os meus tokens se esgotaram; parece que deixei nas pessoas uma má impressão. Não estou bem de corpo nem de mente; por muito tempo dependi dos outros só para atravessar o dia, e não sei onde está o meu caminho na era da IA.
O que, então, me faz seguir em frente? Talvez apenas os instantes em que uma otimização mostra a sua porcentagem de aceleração, quando as ventoinhas da GPU torcem a escuridão do real num fio de luz verdadeira. Usar a IA para iterar sobre infraestrutura me deu essa chance.
A graça é aquilo que está além do que uma pessoa consegue alcançar sozinha. Creio profundamente que iterar sobre infraestrutura com IA é uma forma dessa graça: nesta época, os ganhos de ponta a ponta tornaram-se para mim um caminho importante para receber a graça e firmar a mente.
O ensaio
O ensaio completo está no site de Amodei. É obra dele, por isso esta página leva até ele em vez de copiá-lo.
In oktober 2024 publiceerde Dario Amodei, topman van Anthropic, het lange essay Machines of Loving Grace. Waar de meeste gesprekken over krachtige AI, ook de zijne, over risico’s gaan, beschrijft hij hier zo concreet mogelijk wat er goed zou kunnen gaan als de risico’s beheerst worden. Hij stelt zich een AI voor die op de meeste terreinen slimmer is dan een Nobelprijswinnaar, dagenlang zelfstandig kan werken en in miljoenen kopieën tegelijk kan draaien: ‘een land van genieën in een datacenter’. Wat haar beperkt, betoogt hij, is niet intelligentie maar de wereld eromheen: experimenten die tijd kosten, gegevens die nog niet bestaan, natuurwetten en mensen. Daarna loopt hij vijf terreinen langs: biologie en geneeskunde, waar vijftig tot honderd jaar vooruitgang in vijf of tien jaar zou kunnen komen; neurowetenschap en geestelijke gezondheid; economische ontwikkeling en armoede; vrede en bestuur; en werk en zingeving. De titel komt uit een gedicht van Richard Brautigan uit 1967.
Waarom ik aan AI-infrastructuur werk
Ik ben niet goed in algoritmen voor machine learning. Lange tijd worstelde ik met de angst over mijn loopbaan die AI met zich meebracht, en op een dag waren mijn ideeën en mijn tokens allebei op; ik heb mensen geloof ik geen goede indruk gegeven. Ik ben naar lichaam en geest niet gezond; lange tijd was ik van anderen afhankelijk om de dag door te komen, en ik weet niet waar mijn weg ligt in het tijdperk van AI.
Wat houdt me dan gaande? Misschien alleen de momenten waarop een optimalisatie haar percentage versnelling toont, wanneer de ventilatoren van de GPU de duisternis van het werkelijke tot een draad van echt licht draaien. Infrastructuur met AI verbeteren heeft me die kans gegeven.
Genade is wat verder reikt dan wat een mens op eigen kracht kan bereiken. Ik geloof diep dat het verbeteren van infrastructuur met AI een vorm van die genade is: in deze tijd is end-to-endwinst voor mij een belangrijke weg geworden om genade te ontvangen en innerlijk houvast te vinden.
Het essay
Het volledige essay staat op de website van Amodei. Het is zijn werk; deze pagina verwijst ernaar in plaats van het over te nemen.
Tháng 10 năm 2024, Dario Amodei, giám đốc điều hành của Anthropic, công bố bài tiểu luận dài Machines of Loving Grace. Trong khi phần lớn những gì người ta nói về AI mạnh, kể cả chính ông, đều xoay quanh rủi ro, thì ở đây ông mô tả cụ thể nhất có thể những gì có thể diễn ra tốt đẹp nếu các rủi ro được kiểm soát. Ông hình dung một AI thông minh hơn người đoạt giải Nobel trong hầu hết các lĩnh vực, có thể tự làm việc nhiều ngày liền và chạy hàng triệu bản sao cùng lúc: “một đất nước của những thiên tài trong một trung tâm dữ liệu”. Theo ông, điều giới hạn nó không phải trí tuệ mà là thế giới xung quanh: những thí nghiệm cần thời gian, những dữ liệu chưa tồn tại, các định luật vật lý, và con người. Sau đó ông lần lượt xem xét năm lĩnh vực: sinh học và y học, nơi năm mươi đến một trăm năm tiến bộ có thể đến trong năm hay mười năm; khoa học thần kinh và sức khỏe tâm thần; phát triển kinh tế và nghèo đói; hòa bình và quản trị; công việc và ý nghĩa. Nhan đề lấy từ một bài thơ năm 1967 của Richard Brautigan.
Vì sao tôi làm hạ tầng AI
Tôi không giỏi các thuật toán học máy. Suốt một thời gian dài tôi vật lộn với nỗi lo về sự nghiệp mà AI mang đến, và một ngày kia cả ý tưởng lẫn token của tôi đều cạn; có lẽ tôi đã để lại cho mọi người một ấn tượng không tốt. Tôi không khỏe cả về thể xác lẫn tinh thần; suốt một thời gian dài tôi phải dựa vào người khác chỉ để sống qua từng ngày, và tôi không biết con đường của mình nằm ở đâu trong thời đại AI.
Vậy điều gì giữ cho tôi tiếp tục bước đi? Có lẽ chỉ là những khoảnh khắc khi một tối ưu hóa hiện ra phần trăm tăng tốc của nó, khi quạt của GPU xoắn bóng tối của hiện thực thành một sợi ánh sáng thật. Dùng AI để cải tiến hạ tầng đã cho tôi cơ hội ấy.
Ân điển là điều nằm ngoài những gì một người có thể tự mình đạt tới. Tôi tin sâu sắc rằng cải tiến hạ tầng cùng AI là một hình thức của ân điển ấy: trong thời đại này, lợi ích đầu-cuối đã trở thành một con đường quan trọng để tôi đón nhận ân điển và giữ cho tâm trí vững vàng.
Bài tiểu luận
Toàn văn bài tiểu luận nằm trên trang của Amodei. Đó là tác phẩm của ông, nên trang này dẫn liên kết thay vì sao chép.
ฉันไม่เก่งเรื่องอัลกอริทึมการเรียนรู้ของเครื่อง เป็นเวลานานที่ฉันต่อสู้กับความกังวลเรื่องอนาคตการงานที่ AI นำมา และวันหนึ่งทั้งความคิดและโทเค็นของฉันก็หมดลง ดูเหมือนฉันจะทิ้งความประทับใจที่ไม่ดีไว้กับผู้คน ฉันไม่สบายทั้งกายและใจ เป็นเวลานานที่ฉันต้องพึ่งพาผู้อื่นเพียงเพื่อให้ผ่านแต่ละวันไปได้ และฉันไม่รู้ว่าทางของฉันอยู่ที่ใดในยุคของ AI
แล้วอะไรเล่าที่ทำให้ฉันยังก้าวต่อไป? บางทีอาจเป็นเพียงช่วงเวลาที่การปรับแต่งประสิทธิภาพแสดงเปอร์เซ็นต์ความเร็วที่เพิ่มขึ้น เมื่อพัดลมของ GPU บิดความมืดของความจริงให้กลายเป็นเส้นด้ายแห่งแสงแท้ การใช้ AI พัฒนาโครงสร้างพื้นฐานได้มอบโอกาสนั้นให้ฉัน
พระคุณคือสิ่งที่อยู่เหนือกว่าสิ่งที่คนคนหนึ่งจะไปถึงได้ด้วยตนเอง ฉันเชื่ออย่างลึกซึ้งว่าการพัฒนาโครงสร้างพื้นฐานไปกับ AI เป็นรูปแบบหนึ่งของพระคุณนั้น: ในยุคนี้ ผลลัพธ์แบบต้นจนจบได้กลายเป็นหนทางสำคัญที่ฉันจะรับพระคุณและทำให้จิตใจมั่นคง
अक्टूबर 2024 में Anthropic के मुख्य कार्यकारी डारियो अमोदेई ने लंबा निबंध Machines of Loving Grace प्रकाशित किया। जहाँ शक्तिशाली AI के बारे में ज़्यादातर बातें, उनकी अपनी भी, जोखिम के बारे में होती हैं, वहीं यहाँ वे जितना हो सके ठोस रूप में बताते हैं कि अगर जोखिमों को सँभाल लिया जाए तो क्या-क्या अच्छा हो सकता है। वे ऐसे AI की कल्पना करते हैं जो अधिकांश क्षेत्रों में किसी नोबेल पुरस्कार विजेता से अधिक बुद्धिमान हो, कई दिनों तक अपने आप काम कर सके और एक साथ लाखों प्रतियों में चल सके: “एक डेटा सेंटर में प्रतिभाओं का देश”। उनका तर्क है कि उसे सीमित करने वाली चीज़ बुद्धि नहीं, बल्कि उसके आसपास की दुनिया है: समय लेने वाले प्रयोग, वह डेटा जो अभी मौजूद ही नहीं है, भौतिक नियम, और लोग। फिर वे बारी-बारी से पाँच क्षेत्रों को लेते हैं: जीवविज्ञान और चिकित्सा, जहाँ पचास से सौ साल की प्रगति पाँच या दस साल में आ सकती है; तंत्रिका विज्ञान और मानसिक स्वास्थ्य; आर्थिक विकास और गरीबी; शांति और शासन; और काम और अर्थ। शीर्षक रिचर्ड ब्रॉटिगन की 1967 की एक कविता से लिया गया है।
मैं AI इन्फ्रास्ट्रक्चर पर क्यों काम करता हूँ
मैं मशीन लर्निंग के एल्गोरिदम में अच्छा नहीं हूँ। लंबे समय तक मैं AI से आई अपने करियर की चिंता से जूझता रहा, और एक दिन मेरे विचार और मेरे टोकन दोनों खत्म हो गए; शायद मैंने लोगों पर अच्छी छाप नहीं छोड़ी। मैं न शरीर से स्वस्थ हूँ, न मन से; लंबे समय तक मैं बस दिन काटने के लिए भी दूसरों पर निर्भर रहा, और मुझे नहीं पता कि AI के युग में मेरी राह कहाँ है।
तो फिर मुझे आगे क्या बढ़ाता है? शायद सिर्फ़ वे पल, जब कोई ऑप्टिमाइज़ेशन अपनी गति-वृद्धि का प्रतिशत दिखाता है, जब GPU के पंखे यथार्थ के अँधेरे को बटकर सच्ची रोशनी का एक धागा बना देते हैं। AI के साथ इन्फ्रास्ट्रक्चर को बेहतर बनाने ने मुझे यह अवसर दिया है।
अनुग्रह वह है जो उससे परे है जहाँ तक कोई व्यक्ति अपने बल पर पहुँच सकता है। मेरा गहरा विश्वास है कि AI के साथ इन्फ्रास्ट्रक्चर को बेहतर बनाना उसी अनुग्रह का एक रूप है: इस युग में एंड-टू-एंड लाभ मेरे लिए अनुग्रह पाने और मन को स्थिर रखने का एक महत्वपूर्ण मार्ग बन गया है।
निबंध
पूरा निबंध अमोदेई की वेबसाइट पर है। यह उनकी रचना है, इसलिए यह पृष्ठ उसकी नकल करने के बजाय उसका लिंक देता है।
באוקטובר 2024 פרסם דריו אמודיי, מנכ״ל Anthropic, את המסה הארוכה Machines of Loving Grace. בעוד שרוב הדיבור על בינה מלאכותית חזקה, כולל שלו עצמו, עוסק בסיכונים, כאן הוא משרטט באופן מוחשי ככל האפשר מה עשוי להצליח אם נצליח לרסן את הסיכונים. הוא מדמיין בינה מלאכותית חכמה מזוכה פרס נובל ברוב התחומים, המסוגלת לעבוד בעצמה ימים שלמים ולרוץ במיליוני עותקים בו־זמנית: „ארץ של גאונים במרכז נתונים”. מה שמגביל אותה, הוא טוען, אינו האינטליגנציה אלא העולם שסביבה: ניסויים שדורשים זמן, נתונים שעדיין אינם קיימים, חוקי הטבע ובני אדם. לאחר מכן הוא עובר בזה אחר זה על חמישה תחומים: ביולוגיה ורפואה, שבהן חמישים עד מאה שנות התקדמות עשויות להגיע בתוך חמש או עשר; מדעי המוח ובריאות הנפש; פיתוח כלכלי ועוני; שלום וממשל; ועבודה ומשמעות. הכותרת לקוחה משירו של ריצ׳רד בראוטיגן משנת 1967.
למה אני עובד על תשתיות בינה מלאכותית
אני לא טוב באלגוריתמים של למידת מכונה. זמן רב נאבקתי בחרדה לגבי עתידי המקצועי שהביאה הבינה המלאכותית, ויום אחד נגמרו לי גם הרעיונות וגם הטוקנים; נראה שהשארתי אצל אנשים רושם לא טוב. אינני בריא בגוף ובנפש; זמן רב הייתי תלוי באחרים רק כדי לעבור את היום, ואינני יודע היכן דרכי בעידן הבינה המלאכותית.
מה, אם כן, ממשיך להניע אותי? אולי רק הרגעים שבהם אופטימיזציה מציגה את אחוז ההאצה שלה, כשמאווררי ה־GPU שוזרים את חשכת המציאות לחוט של אור אמיתי. פיתוח תשתיות בעזרת בינה מלאכותית נתן לי את ההזדמנות הזאת.
חסד הוא מה שמעבר למה שאדם יכול להשיג בכוחות עצמו. אני מאמין עמוקות ששיפור תשתיות יחד עם בינה מלאכותית הוא צורה אחת של אותו חסד: בעידן הזה, רווחים מקצה לקצה הפכו עבורי לדרך חשובה לקבל חסד ולמצוא יציבות בנפש.
המסה
המסה המלאה נמצאת באתר של אמודיי. זו יצירתו, ולכן הדף הזה מקשר אליה במקום להעתיק אותה.
The Torre Girona chapel and the MareNostrum supercomputing center, Barcelona
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 in the chapel, 2017.
On the north campus of the Polytechnic University of Catalonia in Barcelona stands the chapel of Torre Girona. It no longer holds services. Since 2005 it has housed MareNostrum, the main supercomputer of the Barcelona Supercomputing Center, in a glass room built inside the nave.
The upper floor of the glass room, under the chapel’s arches (MareNostrum III).
The name is the Roman one for the Mediterranean: mare nostrum, our sea. The machine was first booted on 12 April 2005 and has been renewed several times since; the fifth generation, a EuroHPC pre-exascale system, went into service in December 2023.
MareNostrum 5.
In the scene on this page the machine stands where the altar was, and the chapel sits on a plinth like the other buildings. The near wall is drawn as an open arcade, so that the racks and the lit edges of the glass room show from outside.
The author has used more than 10,000 NVIDIA H100 GPU-hours on MareNostrum for research and development.
Die Kapelle Torre Girona und das Supercomputing-Zentrum MareNostrum, Barcelona
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 in der Kapelle, 2017.
Auf dem Nordcampus der Polytechnischen Universität Katalonien in Barcelona steht die Kapelle Torre Girona. Gottesdienste finden dort nicht mehr statt. Seit 2005 steht in ihr MareNostrum, der Hauptrechner des Barcelona Supercomputing Center, in einem Glasraum, der in das Kirchenschiff gebaut wurde.
Das Obergeschoss des Glasraums unter den Bögen der Kapelle (MareNostrum III).
Der Name ist der römische für das Mittelmeer: mare nostrum, unser Meer. Die Maschine wurde am 12. April 2005 zum ersten Mal gestartet und seitdem mehrfach erneuert; die fünfte Generation, ein Pre-Exascale-System von EuroHPC, ging im Dezember 2023 in Betrieb.
MareNostrum 5.
In der Szene auf dieser Seite steht die Maschine dort, wo der Altar war, und die Kapelle sitzt wie die anderen Bauten auf einem Sockel. Die vordere Wand ist als offene Arkade gezeichnet, damit die Racks und die leuchtenden Kanten des Glasraums von außen zu sehen sind.
Der Autor hat auf MareNostrum mehr als 10.000 NVIDIA-H100-GPU-Stunden für Forschung und Entwicklung genutzt.
La chapelle Torre Girona et le centre de supercalcul MareNostrum, Barcelone
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 dans la chapelle, 2017.
Sur le campus nord de l’Université polytechnique de Catalogne, à Barcelone, se dresse la chapelle Torre Girona. On n’y célèbre plus d’offices. Depuis 2005, elle abrite MareNostrum, le principal calculateur du Barcelona Supercomputing Center, dans une salle de verre construite dans la nef.
L’étage supérieur de la salle de verre sous les arcs de la chapelle (MareNostrum III).
Le nom est celui que les Romains donnaient à la Méditerranée : mare nostrum, notre mer. La machine a démarré pour la première fois le 12 avril 2005 et a été renouvelée plusieurs fois depuis ; la cinquième génération, un système pré-exaflopique d’EuroHPC, est entrée en service en décembre 2023.
MareNostrum 5.
Dans la scène de cette page, la machine se tient là où était l’autel, et la chapelle repose sur un socle comme les autres bâtiments. Le mur de façade est dessiné comme une arcade ouverte, pour que l’on voie de l’extérieur les baies et les arêtes lumineuses de la salle de verre.
L’auteur a utilisé plus de 10 000 heures de GPU NVIDIA H100 sur MareNostrum pour la recherche et le développement.
La capilla de Torre Girona y el Centro de Supercomputación MareNostrum, Barcelona
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 en la capilla, 2017.
En el campus norte de la Universidad Politécnica de Cataluña, en Barcelona, se alza la capilla de Torre Girona. Ya no se celebra culto en ella. Desde 2005 alberga MareNostrum, el ordenador principal del Barcelona Supercomputing Center, en una sala de cristal construida dentro de la nave.
La planta superior de la sala de cristal bajo los arcos de la capilla (MareNostrum III).
El nombre es el que los romanos daban al Mediterráneo: mare nostrum, nuestro mar. La máquina arrancó por primera vez el 12 de abril de 2005 y desde entonces se ha renovado varias veces; la quinta generación, un sistema preexascala de EuroHPC, entró en servicio en diciembre de 2023.
MareNostrum 5.
En la escena de esta página, la máquina está donde estaba el altar, y la capilla descansa sobre un zócalo como los demás edificios. La pared frontal se dibuja como una arcada abierta, para que desde fuera se vean los racks y las aristas luminosas de la sala de cristal.
El autor ha utilizado más de 10 000 horas de GPU NVIDIA H100 en MareNostrum para investigación y desarrollo.
Sacellum Torre Girona et centrum supercomputatorium MareNostrum, Barcinone
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 in sacello, anno 2017.
In campo septentrionali Universitatis Polytechnicae Cataloniae Barcinone stat sacellum Torre Girona. Sacra ibi iam non fiunt. Ab anno 2005 MareNostrum, machina praecipua Barcelona Supercomputing Center, in eo stat, in conclavi vitreo quod in navi aedificatum est.
Tabulatum superius conclavis vitrei sub arcubus sacelli (MareNostrum III).
Nomen est quo Romani mare Mediterraneum appellabant: mare nostrum. Machina die duodecimo mensis Aprilis anni 2005 primum incensa est et saepius postea renovata; quinta eius generatio, systema EuroHPC prae-exascale, mense Decembri anni 2023 opus coepit.
MareNostrum 5.
In scaena huius paginae machina stat ubi altare fuit, et sacellum, ut cetera aedificia, in basi positum est. Paries anterior ut porticus aperta pingitur, ut armaria et margines lucentes conclavis vitrei extrinsecus videantur.
Auctor plus quam decem milia horarum GPU NVIDIA H100 in MareNostrum ad investigationem et progressum adhibuit.
Капелла Торре-Жирона и суперкомпьютерный центр MareNostrum, Барселона
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 в капелле, 2017 год.
В северном кампусе Политехнического университета Каталонии в Барселоне стоит капелла Торре-Жирона. Службы в ней больше не проводятся. С 2005 года в ней размещается MareNostrum, главный суперкомпьютер Барселонского суперкомпьютерного центра, — в стеклянном зале, построенном внутри нефа.
Верхний этаж стеклянного зала под арками капеллы (MareNostrum III).
Название — римское имя Средиземного моря: mare nostrum, «наше море». Машину впервые запустили 12 апреля 2005 года, и с тех пор её несколько раз обновляли; пятое поколение, предэкзафлопсная система EuroHPC, вступило в строй в декабре 2023 года.
MareNostrum 5.
В сцене на этой странице машина стоит там, где был алтарь, а капелла, как и другие здания, стоит на постаменте. Ближняя стена нарисована открытой аркадой, чтобы стойки и светящиеся грани стеклянного зала были видны снаружи.
Автор использовал на MareNostrum более 10 000 GPU-часов NVIDIA H100 для исследований и разработки.
A capela Torre Girona e o centro de supercomputação MareNostrum, Barcelona
Dezhen Lu, Claude Opus 5.5
O MareNostrum 4 na capela, 2017.
No campus norte da Universidade Politécnica da Catalunha, em Barcelona, ergue-se a capela de Torre Girona. Já não se celebram missas ali. Desde 2005 ela abriga o MareNostrum, o principal supercomputador do Barcelona Supercomputing Center, numa sala de vidro construída dentro da nave.
O andar superior da sala de vidro, sob os arcos da capela (MareNostrum III).
O nome é o que os romanos davam ao Mediterrâneo: mare nostrum, o nosso mar. A máquina foi ligada pela primeira vez em 12 de abril de 2005 e renovada várias vezes desde então; a quinta geração, um sistema pré-exascale da EuroHPC, entrou em operação em dezembro de 2023.
MareNostrum 5.
Na cena desta página, a máquina está onde ficava o altar, e a capela assenta num pedestal como os outros edifícios. A parede da frente é desenhada como uma arcada aberta, para que os racks e as arestas iluminadas da sala de vidro se vejam de fora.
O autor usou mais de 10.000 horas de GPU NVIDIA H100 no MareNostrum em pesquisa e desenvolvimento.
De kapel Torre Girona en het supercomputercentrum MareNostrum, Barcelona
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 in de kapel, 2017.
Op de noordcampus van de Polytechnische Universiteit van Catalonië in Barcelona staat de kapel van Torre Girona. Er worden geen diensten meer gehouden. Sinds 2005 staat er MareNostrum, de belangrijkste supercomputer van het Barcelona Supercomputing Center, in een glazen ruimte die in het schip is gebouwd.
De bovenverdieping van de glazen ruimte, onder de bogen van de kapel (MareNostrum III).
De naam is die van de Romeinen voor de Middellandse Zee: mare nostrum, onze zee. De machine werd op 12 april 2005 voor het eerst opgestart en is sindsdien meermaals vernieuwd; de vijfde generatie, een pre-exascalesysteem van EuroHPC, werd in december 2023 in gebruik genomen.
MareNostrum 5.
In de scène op deze pagina staat de machine waar het altaar was, en staat de kapel net als de andere gebouwen op een sokkel. De voorste muur is als open arcade getekend, zodat de racks en de oplichtende randen van de glazen ruimte van buitenaf te zien zijn.
De auteur heeft op MareNostrum meer dan 10.000 NVIDIA H100-GPU-uren gebruikt voor onderzoek en ontwikkeling.
Nhà nguyện Torre Girona và Trung tâm Siêu máy tính MareNostrum, Barcelona
Dezhen Lu, Claude Opus 5.5
MareNostrum 4 trong nhà nguyện, năm 2017.
Trong khuôn viên phía bắc của Đại học Bách khoa Catalunya ở Barcelona có nhà nguyện Torre Girona. Nơi đây không còn cử hành lễ nữa. Từ năm 2005, nhà nguyện là nơi đặt MareNostrum, siêu máy tính chính của Trung tâm Siêu máy tính Barcelona, trong một căn phòng kính dựng bên trong gian chính.
Tầng trên của căn phòng kính, dưới những mái vòm của nhà nguyện (MareNostrum III).
Cái tên là tên người La Mã gọi Địa Trung Hải: mare nostrum, “biển của chúng ta”. Cỗ máy được khởi động lần đầu ngày 12 tháng 4 năm 2005 và từ đó đã được nâng cấp nhiều lần; thế hệ thứ năm, một hệ thống tiền exascale của EuroHPC, đi vào hoạt động tháng 12 năm 2023.
MareNostrum 5.
Trong khung cảnh ở trang này, cỗ máy đứng ở chỗ bàn thờ ngày trước, và nhà nguyện đặt trên một bệ như những công trình khác. Bức tường phía trước được vẽ thành một dãy vòm để ngỏ, để từ bên ngoài vẫn thấy các tủ rack và những cạnh phát sáng của căn phòng kính.
Tác giả đã dùng hơn 10.000 giờ GPU NVIDIA H100 trên MareNostrum cho nghiên cứu và phát triển.
तोर्रे जिरोना चैपल और MareNostrum सुपरकंप्यूटिंग केंद्र, बार्सिलोना
Dezhen Lu, Claude Opus 5.5
चैपल में MareNostrum 4, 2017।
बार्सिलोना में कैटालोनिया पॉलिटेक्निक विश्वविद्यालय के उत्तरी परिसर में तोर्रे जिरोना का चैपल खड़ा है। अब वहाँ आराधना नहीं होती। 2005 से इसमें बार्सिलोना सुपरकंप्यूटिंग सेंटर का मुख्य सुपरकंप्यूटर MareNostrum रखा है, चैपल के मध्य भाग में बने एक काँच के कमरे में।
चैपल की मेहराबों के नीचे काँच के कमरे की ऊपरी मंज़िल (MareNostrum III)।
यह नाम भूमध्य सागर का रोमन नाम है: mare nostrum, यानी “हमारा सागर”। मशीन पहली बार 12 अप्रैल 2005 को चालू हुई और तब से कई बार नवीनीकृत की गई है; पाँचवीं पीढ़ी, EuroHPC की एक प्री-एक्सास्केल प्रणाली, दिसंबर 2023 में सेवा में आई।
MareNostrum 5।
इस पृष्ठ के दृश्य में मशीन वहाँ खड़ी है जहाँ पहले वेदी थी, और चैपल दूसरी इमारतों की तरह एक चबूतरे पर रखा है। सामने की दीवार खुली मेहराबों की कतार के रूप में बनाई गई है, ताकि रैक और काँच के कमरे के चमकते किनारे बाहर से दिखें।
लेखक ने शोध और विकास के लिए MareNostrum पर 10,000 से अधिक NVIDIA H100 GPU-घंटे इस्तेमाल किए हैं।
בקמפוס הצפוני של האוניברסיטה הפוליטכנית של קטלוניה בברצלונה עומדת הקפלה של טורה ז׳ירונה. כבר לא מתקיימות בה תפילות. מאז 2005 שוכן בה MareNostrum, מחשב־העל הראשי של מרכז מחשוב־העל של ברצלונה, בתוך חדר זכוכית שנבנה באולם התווך.
הקומה העליונה של חדר הזכוכית, מתחת לקשתות הקפלה (MareNostrum III).
השם הוא שמו הרומי של הים התיכון: mare nostrum, „הים שלנו”. המכונה הופעלה לראשונה ב־12 באפריל 2005 וחודשה מאז כמה פעמים; הדור החמישי, מערכת טרום־אקסה של EuroHPC, נכנס לשירות בדצמבר 2023.
MareNostrum 5.
בסצנה שבדף הזה המכונה עומדת במקום שבו היה המזבח, והקפלה מונחת על בסיס כמו שאר המבנים. הקיר הקרוב מצויר כארקדה פתוחה, כדי שהארונות והשוליים המוארים של חדר הזכוכית ייראו מבחוץ.
המחבר השתמש ביותר מ־10,000 שעות GPU של NVIDIA H100 ב־MareNostrum למחקר ופיתוח.
arXiv 2611.05150 · Submitted to MLSys · 16 pages, 3 figures, 10 tables. Appendix included
Abstract
Looped language models apply one block of layers many times per token, so a serving engine pays every per-layer overhead once per loop; we measure serving for Ouro and Huginn-0125 in an open-source looped-model engine and grade our optimizations from bitwise-identical changes to those that require an accuracy check. At batch 1, fusing all loop submissions leaves Ouro throughput at 0.99x; the 5689 small kernels per token are the real cost, and fusing them behind rounding fences raises throughput 1.11-1.59x while 1014/1014 requests remain identical. An approximate fusion is no faster and misses a one-point GSM8K rule on 87 questions (61 and 62 correct versus 64), so we do not ship it. Huginn's released defaults avoid dead KV slots, reuse prefix-cache pages, and serve 1.77-3.16x faster than the base engine; Ouro reaches 1.65-2.15x and agent sessions 1.91-3.21x, while Huginn's noise and prefix-reuse changes stay within one point on all 1319 Huginn test questions; Huginn's packed prefill, -0.61 points on the same questions, meets that rule only on its point estimate. Position-keyed noise makes CPU cache reuse exact but still leaves 20 of 128 GPU full hits different, while loop fusion delays prefill admission and a 64-graph cache prevents eager fallbacks. Under prefill-decode disaggregation ExactLoop reaches up to 3.3x the base throughput with peer error handling disabled, and 3.1x in two of three runs with it enabled. Its transfer budget, sized to hold one request's KV in flight, cuts the commit tail from about one second under the earlier budget to 16 ms (base: 2.8 s), removing the polling rounds that the engine's interval would otherwise impose.
arXiv 2610.41825 · Submitted to MLSys · 11 pages of main text (37 pages including appendix), 5 figures, 37 tables
Abstract
A looped language model applies one block of layers 4 times per token and keeps a key/value (KV) plane for each loop. We study where storage error changes the next-token distribution and how shared-weight error accumulates, using Ouro-1.4B (4 loops) on GSM8K and MATH and Huginn-0125 (32 loops) on GSM8K-derived chains, with dense, simulated quantization. Per unit of error, values carry 94% of the diagonal Fisher mass, but measured KL is dominated by keys with larger quantization errors. Local second-order prediction follows the measured KL of typical sequences, including 136 held-out chains, but misses a few confident rows, and a local second-order factor for shared weights at 32 loops ranges from 1.65 (INT8) to 4.09 (INT4), so it depends on the error direction. A cell-level allocator lowers one-step KL at 25-48% of BF16 bytes, while paired task-accuracy comparisons depend on budget and the 25% gap against K4V2 shrinks to near zero under a post hoc answer-extraction check. A later Ouro-2.6B test retains the key/value split but falsifies measured-cost allocation's advantage over Fisher at 31%; across all 21 ledger registrations, 60 of 176 criteria failed.
Looped language models reuse one block of layers over R recurrent steps but store a key-value (KV) plane for each step. Keeping only the deepest plane shrinks the cache up to R-fold. A natural repair, which we call RIPE, keeps every plane for the W most recent positions and shares the deepest plane elsewhere. This repair acts on the wrong axis: across three looped models, the cost of sharing is set by how far the late loops' planes have converged (ripeness) and by which positions share, not by recency. On Raven-Llama the deep planes sit within the bf16 rounding floor, so no window is needed. On Huginn-0125 a 64-position window removes most of a concentrated replay-error tail (9 to 1 positions over 48 prompts), yet in a real self-speculative decoder the policy is slower and larger than the deepest plane alone and not measurably closer to full-cache decoding. On Ouro-1.4B the prompt lies outside any 16-64-position window: on 24 GSM8K questions with a 2048-token budget, the full cache and a FlashLoop baseline finish 23 answers and RIPE at most 2, although physical storage matches the emulated policy bitwise. Sensitivity and ripening models fitted on Huginn and Raven explain their contrast, and Ouro's own report that prompt-side sharing is harmful explains the third. Separately, bf16 execution-shape effects are as large as the cache effects being measured. Recent-window repair of shared-plane caches should not be adopted without a per-model ripeness test. We release ten pre-registered batches with every failed prediction.
Looped language models such as Ouro reuse one block of layers several times per token. A rollout engine can batch the deeper loops of several response positions and run them only when a deep readout is due. This deferred schedule is fast, but the learner of an on-policy RL algorithm recomputes the sampled tokens' log-probabilities with a dense pass, and the two computations differ in floating point: in FP32 at 4096 tokens about 18% of the sampled tokens miss a 10⁻⁵ same-weight gate, and with bf16 weights nearly every token does. We use a hybrid learner log-probability, ℓ = ℓdense + sg(ℓsched − ℓdense), where sg denotes stop-gradient. Its value is a no-grad replay through the rollout's own schedule, and its gradient is the dense learner's. The replay reproduces the rollout bit for bit in all 167 FP32 and 28 bf16 real-weight cells we ran on three platforms, including rollouts whose batch shrinks at stop tokens, and the gradient has the same FP32 error as other dense implementations in all 102 pre-registered comparisons from 96 to 4096 tokens; the full GRPO loss of a training step matches a float64 reference to 3.1×10⁻⁴. On one GPU, deferral raises rollout throughput by 1.47 to 1.56×, and by 1.18 to 1.54× once every step runs from CUDA graphs, bit for bit. In a two-node GRPO loop, one-wave KV admission and CUDA graphs shorten an update by 2.71× with the gate bitwise at every update, and four nodes keep the update time within 2% at the same per-GPU load. In 100-update runs held-out GSM8K accuracy rose from 23% to 82% with the mixed-depth readout that deferral relies on and from 34% to 90% with the native readout, which reached a higher accuracy in less time. For looped models beyond one node we analyse a ring mapping of layers to GPUs and measure its communication paths.
Weight Placement Is Not Enough: Synchronization in Multi-Node CPU Decoding of Ternary LLMs
Runguo Li, Dezhen Lu
arXiv 2610.28620 · Submitted to MLSys · 15 pages, 9 figures; corrected software-name formatting in the abstract metadata; paper content unchanged
Abstract
Ternary LLMs read only a few gigabytes per decoded token, yet on dual-socket servers with four NUMA nodes the fastest ternary runtime we found uses less than half of the memory bandwidth. Placing each weight on the node of the thread that reads it is necessary but not enough: in a decode engine with node-local weights that reuses ik_llama's kernels, threads on two generations of Xeon servers still spend 20-48% of each token waiting at synchronization points. Per-thread barrier traces show that most of the wait is for the slowest threads of each node; the barriers' own latency (microseconds each, but up to 12% of the token for the smallest model) and serial per-node work are smaller parts. On identical kernels, a Megatron-style column split beats a row-owned split without cross-node reductions in every configuration we measured, and isolating the number of barriers shows that most of its gain comes from its threads waiting less for each other and for their node leaders, and only a minority from having fewer barriers. Where decoding is DRAM-bound, the resulting engine is 1.5-2.1x faster than ik_llama with all cores and 1.5-1.9x faster than ik_llama at its fastest thread count (3.2x and 2.8x for a small model that runs from the L3 cache), at the same accuracy and with less energy per token.
