Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4633 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.8029. Min Act: -0.0000

Data Index: 749118 (The Pile)

Max Activating Token Index: 796

Text #1

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.7257. Min Act: -0.0000

Data Index: 1670134 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.7258. Min Act: -0.0000

Data Index: 1180004 (The Pile)

Max Activating Token Index: 241

Text #3

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6753. Min Act: -0.0000

Data Index: 45952 (The Pile)

Max Activating Token Index: 201

Text #4

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.7102. Min Act: -0.0000

Data Index: 1420783 (The Pile)

Max Activating Token Index: 402

Text #5

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6491. Min Act: -0.0000

Data Index: 978156 (The Pile)

Max Activating Token Index: 264

Text #6

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6022. Min Act: -0.0000

Data Index: 1139569 (The Pile)

Max Activating Token Index: 546

Text #7

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6130. Min Act: -0.0000

Data Index: 34489 (The Pile)

Max Activating Token Index: 188

Text #8

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6460. Min Act: -0.0000

Data Index: 1561295 (The Pile)

Max Activating Token Index: 42

Text #9

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6440. Min Act: -0.0000

Data Index: 1713899 (The Pile)

Max Activating Token Index: 363

Text #10

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.5815. Min Act: -0.0000

Data Index: 165996 (The Pile)

Max Activating Token Index: 393

Text #11

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6047. Min Act: -0.0000

Data Index: 481353 (The Pile)

Max Activating Token Index: 196

Text #12

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6984. Min Act: -0.0000

Data Index: 1252715 (The Pile)

Max Activating Token Index: 915

Text #13

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.5689. Min Act: -0.0000

Data Index: 631647 (The Pile)

Max Activating Token Index: 437

Text #14

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6034. Min Act: -0.0000

Data Index: 936845 (The Pile)

Max Activating Token Index: 775

Text #15

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6101. Min Act: -0.0000

Data Index: 1240593 (The Pile)

Max Activating Token Index: 133

Text #16

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.6618. Min Act: -0.0000

Data Index: 1913263 (The Pile)

Max Activating Token Index: 740

Text #17

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.5638. Min Act: -0.0000

Data Index: 1206548 (The Pile)

Max Activating Token Index: 687

Text #18

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.5264. Min Act: -0.0000

Data Index: 228039 (The Pile)

Max Activating Token Index: 894

Text #19

Max Range: 0.8029. Min Range: -0.8029

Max Act: 0.5946. Min Act: -0.0000

Data Index: 1255223 (The Pile)

Max Activating Token Index: 35