Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 3413 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.5293. Min Act: -0.0000

Data Index: 273688 (The Pile)

Max Activating Token Index: 296

Text #1

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.5290. Min Act: -0.0000

Data Index: 689809 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.3139. Min Act: -0.0000

Data Index: 1126092 (The Pile)

Max Activating Token Index: 189

Text #3

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.3062. Min Act: -0.0000

Data Index: 1650111 (The Pile)

Max Activating Token Index: 691

Text #4

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1776. Min Act: -0.0000

Data Index: 249023 (The Pile)

Max Activating Token Index: 885

Text #5

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2725. Min Act: -0.0000

Data Index: 666816 (The Pile)

Max Activating Token Index: 161

Text #6

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2765. Min Act: -0.0000

Data Index: 1789081 (The Pile)

Max Activating Token Index: 340

Text #7

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.3138. Min Act: -0.0000

Data Index: 1224601 (The Pile)

Max Activating Token Index: 281

Text #8

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2190. Min Act: -0.0000

Data Index: 1578321 (The Pile)

Max Activating Token Index: 196

Text #9

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2268. Min Act: -0.0000

Data Index: 1335333 (The Pile)

Max Activating Token Index: 298

Text #10

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2435. Min Act: -0.0000

Data Index: 1663551 (The Pile)

Max Activating Token Index: 214

Text #11

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1608. Min Act: -0.0000

Data Index: 729923 (The Pile)

Max Activating Token Index: 338

Text #12

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2799. Min Act: -0.0000

Data Index: 1119213 (The Pile)

Max Activating Token Index: 95

Text #13

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.2263. Min Act: -0.0000

Data Index: 1624364 (The Pile)

Max Activating Token Index: 11

Text #14

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1830. Min Act: -0.0000

Data Index: 450126 (The Pile)

Max Activating Token Index: 223

Text #15

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1072. Min Act: -0.0000

Data Index: 1163590 (The Pile)

Max Activating Token Index: 193

Text #16

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1692. Min Act: -0.0000

Data Index: 1112175 (The Pile)

Max Activating Token Index: 810

Text #17

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1959. Min Act: -0.0000

Data Index: 1005225 (The Pile)

Max Activating Token Index: 259

Text #18

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.1835. Min Act: -0.0000

Data Index: 1962570 (The Pile)

Max Activating Token Index: 330

Text #19

Max Range: 1.5293. Min Range: -1.5293

Max Act: 1.0518. Min Act: -0.0000

Data Index: 95908 (The Pile)

Max Activating Token Index: 284