Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 705 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.3865. Min Act: -0.0000

Data Index: 517309 (The Pile)

Max Activating Token Index: 168

Text #1

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2927. Min Act: -0.0000

Data Index: 320202 (The Pile)

Max Activating Token Index: 933

Text #2

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2567. Min Act: -0.0000

Data Index: 239165 (The Pile)

Max Activating Token Index: 108

Text #3

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2409. Min Act: -0.0000

Data Index: 183505 (The Pile)

Max Activating Token Index: 496

Text #4

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2222. Min Act: -0.0000

Data Index: 235765 (The Pile)

Max Activating Token Index: 231

Text #5

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1582. Min Act: -0.0000

Data Index: 298339 (The Pile)

Max Activating Token Index: 691

Text #6

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1869. Min Act: -0.0000

Data Index: 19497 (The Pile)

Max Activating Token Index: 479

Text #7

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1893. Min Act: -0.0000

Data Index: 687338 (The Pile)

Max Activating Token Index: 558

Text #8

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1820. Min Act: -0.0000

Data Index: 1336235 (The Pile)

Max Activating Token Index: 95

Text #9

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2374. Min Act: -0.0000

Data Index: 953616 (The Pile)

Max Activating Token Index: 772

Text #10

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.2417. Min Act: -0.0000

Data Index: 1580068 (The Pile)

Max Activating Token Index: 801

Text #11

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1595. Min Act: -0.0000

Data Index: 1278176 (The Pile)

Max Activating Token Index: 145

Text #12

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1303. Min Act: -0.0000

Data Index: 11822 (The Pile)

Max Activating Token Index: 259

Text #13

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.0753. Min Act: -0.0000

Data Index: 1305580 (The Pile)

Max Activating Token Index: 178

Text #14

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1840. Min Act: -0.0000

Data Index: 1359262 (The Pile)

Max Activating Token Index: 186

Text #15

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1237. Min Act: -0.0000

Data Index: 1681283 (The Pile)

Max Activating Token Index: 347

Text #16

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1833. Min Act: -0.0000

Data Index: 246751 (The Pile)

Max Activating Token Index: 789

Text #17

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1694. Min Act: -0.0000

Data Index: 1563364 (The Pile)

Max Activating Token Index: 485

Text #18

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1712. Min Act: -0.0000

Data Index: 1433274 (The Pile)

Max Activating Token Index: 691

Text #19

Max Range: 1.3865. Min Range: -1.3865

Max Act: 1.1009. Min Act: -0.0000

Data Index: 325133 (The Pile)

Max Activating Token Index: 756