Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2529 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.0336. Min Range: -1.0336

Max Act: 1.0336. Min Act: -0.0000

Data Index: 1130213 (The Pile)

Max Activating Token Index: 305

Text #1

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.9831. Min Act: -0.0000

Data Index: 912310 (The Pile)

Max Activating Token Index: 487

Text #2

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.9131. Min Act: -0.0000

Data Index: 74239 (The Pile)

Max Activating Token Index: 856

Text #3

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.9600. Min Act: -0.0000

Data Index: 113929 (The Pile)

Max Activating Token Index: 1018

Text #4

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.9366. Min Act: -0.0000

Data Index: 199019 (The Pile)

Max Activating Token Index: 453

Text #5

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.7182. Min Act: -0.0000

Data Index: 964123 (The Pile)

Max Activating Token Index: 92

Text #6

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.7675. Min Act: -0.0000

Data Index: 419021 (The Pile)

Max Activating Token Index: 939

Text #7

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.7239. Min Act: -0.0000

Data Index: 364119 (The Pile)

Max Activating Token Index: 320

Text #8

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6638. Min Act: -0.0000

Data Index: 1963139 (The Pile)

Max Activating Token Index: 571

Text #9

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6343. Min Act: -0.0000

Data Index: 20200 (The Pile)

Max Activating Token Index: 490

Text #10

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6770. Min Act: -0.0000

Data Index: 112482 (The Pile)

Max Activating Token Index: 999

Text #11

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6044. Min Act: -0.0000

Data Index: 1023020 (The Pile)

Max Activating Token Index: 365

Text #12

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6640. Min Act: -0.0000

Data Index: 467386 (The Pile)

Max Activating Token Index: 386

Text #13

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6701. Min Act: -0.0000

Data Index: 1402800 (The Pile)

Max Activating Token Index: 407

Text #14

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6196. Min Act: -0.0000

Data Index: 220247 (The Pile)

Max Activating Token Index: 175

Text #15

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.6466. Min Act: -0.0000

Data Index: 1129156 (The Pile)

Max Activating Token Index: 713

Text #16

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.5784. Min Act: -0.0000

Data Index: 1134760 (The Pile)

Max Activating Token Index: 108

Text #17

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.5967. Min Act: -0.0000

Data Index: 60682 (The Pile)

Max Activating Token Index: 999

Text #18

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.5751. Min Act: -0.0000

Data Index: 242481 (The Pile)

Max Activating Token Index: 903

Text #19

Max Range: 1.0336. Min Range: -1.0336

Max Act: 0.5578. Min Act: -0.0000

Data Index: 163703 (The Pile)

Max Activating Token Index: 270