Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2208 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1043. Min Range: -1.1043

Max Act: 1.1043. Min Act: -0.0000

Data Index: 216592 (The Pile)

Max Activating Token Index: 889

Text #1

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9132. Min Act: -0.0000

Data Index: 354627 (The Pile)

Max Activating Token Index: 996

Text #2

Max Range: 1.1043. Min Range: -1.1043

Max Act: 1.0077. Min Act: -0.0000

Data Index: 1960780 (The Pile)

Max Activating Token Index: 323

Text #3

Max Range: 1.1043. Min Range: -1.1043

Max Act: 1.0187. Min Act: -0.0000

Data Index: 1835935 (The Pile)

Max Activating Token Index: 355

Text #4

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9488. Min Act: -0.0000

Data Index: 725093 (The Pile)

Max Activating Token Index: 839

Text #5

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9503. Min Act: -0.0000

Data Index: 1279424 (The Pile)

Max Activating Token Index: 52

Text #6

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9048. Min Act: -0.0000

Data Index: 250440 (The Pile)

Max Activating Token Index: 431

Text #7

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9337. Min Act: -0.0000

Data Index: 717893 (The Pile)

Max Activating Token Index: 563

Text #8

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8803. Min Act: -0.0000

Data Index: 470721 (The Pile)

Max Activating Token Index: 624

Text #9

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8774. Min Act: -0.0000

Data Index: 1349979 (The Pile)

Max Activating Token Index: 800

Text #10

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.9435. Min Act: -0.0000

Data Index: 1964121 (The Pile)

Max Activating Token Index: 349

Text #11

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8320. Min Act: -0.0000

Data Index: 807777 (The Pile)

Max Activating Token Index: 477

Text #12

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8772. Min Act: -0.0000

Data Index: 1763251 (The Pile)

Max Activating Token Index: 306

Text #13

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8784. Min Act: -0.0000

Data Index: 669120 (The Pile)

Max Activating Token Index: 765

Text #14

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8632. Min Act: -0.0000

Data Index: 759525 (The Pile)

Max Activating Token Index: 251

Text #15

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8392. Min Act: -0.0000

Data Index: 710280 (The Pile)

Max Activating Token Index: 331

Text #16

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8730. Min Act: -0.0000

Data Index: 799109 (The Pile)

Max Activating Token Index: 789

Text #17

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8909. Min Act: -0.0000

Data Index: 1789566 (The Pile)

Max Activating Token Index: 716

Text #18

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.8650. Min Act: -0.0000

Data Index: 1071341 (The Pile)

Max Activating Token Index: 279

Text #19

Max Range: 1.1043. Min Range: -1.1043

Max Act: 0.7988. Min Act: -0.0000

Data Index: 1981590 (The Pile)

Max Activating Token Index: 390