Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4353 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1447. Min Act: -0.0000

Data Index: 848531 (The Pile)

Max Activating Token Index: 452

Text #1

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1315. Min Act: -0.0000

Data Index: 1903441 (The Pile)

Max Activating Token Index: 395

Text #2

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1206. Min Act: -0.0000

Data Index: 1069903 (The Pile)

Max Activating Token Index: 570

Text #3

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1217. Min Act: -0.0000

Data Index: 1176172 (The Pile)

Max Activating Token Index: 325

Text #4

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1207. Min Act: -0.0000

Data Index: 1787361 (The Pile)

Max Activating Token Index: 45

Text #5

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1049. Min Act: -0.0000

Data Index: 777397 (The Pile)

Max Activating Token Index: 617

Text #6

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1151. Min Act: -0.0000

Data Index: 1891456 (The Pile)

Max Activating Token Index: 858

Text #7

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1127. Min Act: -0.0000

Data Index: 346885 (The Pile)

Max Activating Token Index: 861

Text #8

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1104. Min Act: -0.0000

Data Index: 773415 (The Pile)

Max Activating Token Index: 773

Text #9

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1063. Min Act: -0.0000

Data Index: 1479546 (The Pile)

Max Activating Token Index: 523

Text #10

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1030. Min Act: -0.0000

Data Index: 483110 (The Pile)

Max Activating Token Index: 823

Text #11

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1061. Min Act: -0.0000

Data Index: 1712406 (The Pile)

Max Activating Token Index: 318

Text #12

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1090. Min Act: -0.0000

Data Index: 1641319 (The Pile)

Max Activating Token Index: 339

Text #13

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1090. Min Act: -0.0000

Data Index: 483898 (The Pile)

Max Activating Token Index: 474

Text #14

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1081. Min Act: -0.0001

Data Index: 1544479 (The Pile)

Max Activating Token Index: 701

Text #15

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1117. Min Act: -0.0000

Data Index: 1452533 (The Pile)

Max Activating Token Index: 623

Text #16

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1077. Min Act: -0.0000

Data Index: 587263 (The Pile)

Max Activating Token Index: 162

Text #17

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1105. Min Act: -0.0000

Data Index: 1297402 (The Pile)

Max Activating Token Index: 572

Text #18

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1033. Min Act: -0.0000

Data Index: 1409680 (The Pile)

Max Activating Token Index: 583

Text #19

Max Range: 0.1447. Min Range: -0.1447

Max Act: 0.1077. Min Act: -0.0000

Data Index: 1451773 (The Pile)

Max Activating Token Index: 625