Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 1420 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1894. Min Act: -0.0000

Data Index: 1058687 (The Pile)

Max Activating Token Index: 944

Text #1

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1707. Min Act: -0.0000

Data Index: 1871459 (The Pile)

Max Activating Token Index: 818

Text #2

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1745. Min Act: -0.0000

Data Index: 18667 (The Pile)

Max Activating Token Index: 499

Text #3

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1839. Min Act: -0.0000

Data Index: 1485411 (The Pile)

Max Activating Token Index: 709

Text #4

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1656. Min Act: -0.0000

Data Index: 1921037 (The Pile)

Max Activating Token Index: 164

Text #5

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1599. Min Act: -0.0000

Data Index: 1733929 (The Pile)

Max Activating Token Index: 492

Text #6

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1267. Min Act: -0.0000

Data Index: 201670 (The Pile)

Max Activating Token Index: 515

Text #7

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1462. Min Act: -0.0000

Data Index: 710483 (The Pile)

Max Activating Token Index: 1004

Text #8

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1369. Min Act: -0.0000

Data Index: 889576 (The Pile)

Max Activating Token Index: 298

Text #9

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1350. Min Act: -0.0000

Data Index: 559471 (The Pile)

Max Activating Token Index: 898

Text #10

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1313. Min Act: -0.0000

Data Index: 314877 (The Pile)

Max Activating Token Index: 399

Text #11

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1309. Min Act: -0.0000

Data Index: 744955 (The Pile)

Max Activating Token Index: 895

Text #12

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1326. Min Act: -0.0000

Data Index: 1491075 (The Pile)

Max Activating Token Index: 192

Text #13

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1190. Min Act: -0.0000

Data Index: 812971 (The Pile)

Max Activating Token Index: 518

Text #14

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1295. Min Act: -0.0000

Data Index: 483377 (The Pile)

Max Activating Token Index: 287

Text #15

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1126. Min Act: -0.0000

Data Index: 717231 (The Pile)

Max Activating Token Index: 654

Text #16

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1200. Min Act: -0.0000

Data Index: 1522423 (The Pile)

Max Activating Token Index: 680

Text #17

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1189. Min Act: -0.0000

Data Index: 1077178 (The Pile)

Max Activating Token Index: 254

Text #18

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1181. Min Act: -0.0000

Data Index: 1828160 (The Pile)

Max Activating Token Index: 876

Text #19

Max Range: 0.1894. Min Range: -0.1894

Max Act: 0.1084. Min Act: -0.0000

Data Index: 830095 (The Pile)

Max Activating Token Index: 377